Network management fundamentals nov 2006

ii

Network Management Fundamentals
Alexander Clemm, Ph.D.
Copyright© 2007 Cisco Systems, Inc.
Published by:
Cisco Press
800 East 96th Street
Indianapolis, IN 46240 USA
All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying, recording, or by any information storage and retrieval system, without written permission from the publisher, except for the inclusion of brief quotations in a review.
Printed in the United States of America 1 2 3 4 5 6 7 8 9 0
First Printing November 2006
LIBRARY OF CONGRESS CATALOG CARD NUMBER: 2004110268
ISBN: 1-58720-137-2

Warning and Disclaimer
This book is designed to provide information about network management. Every effort has been made to make this book as complete
and as accurate as possible, but no warranty or fitness is implied.
The information is provided on an “as is” basis. The authors, Cisco Press, and Cisco Systems, Inc., shall have neither liability nor
responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book or from
the use of the discs or programs that may accompany it.
The opinions expressed in this book belong to the author and are not necessarily those of Cisco Systems, Inc.

Corporate and Government Sales
Cisco Press offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact: U.S. Corporate and Government Sales 1-800-382-3419
For sales outside of the U.S. please contact:

International Sales

1-317-581-3793



Feedback Information
At Cisco Press, our goal is to create in-depth technical books of the highest quality and value. Each book is crafted with care and precision, undergoing rigorous development that involves the unique expertise of members from the professional technical community.
Readers’ feedback is a natural continuation of this process. If you have any comments regarding how we could improve the quality
of this book or otherwise alter it to better suit your needs, you can contact us through e-mail at Please
make sure to include the book title and ISBN in your message.
We greatly appreciate your assistance.

Trademark Acknowledgments
All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Cisco Press
or Cisco Systems, Inc., cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark.


iii

Publisher: Paul Boger

Cisco Representative: Anthony Wolfenden

Executive Editor: Mary Beth Ray

Cisco Press Program Manager: Jeff Brady

Managing Editor: Patrick Kanouse

Technical Editors: Prakash Bettadapur, David M. Kurtiak, Lundy Lewis

Development Editor: Betsey Henkels
Project Editor: Tonya Simpson
Copy Editor: Krista Hansing Editorial Services, Inc.
Team Coordinator: Vanessa Evans
Book and Cover Designer: Louisa Adair
Compositor: Mark Shirar
Indexer: Larry Sweazy


iv

About the Author
Dr. Alexander Clemm, Ph.D. is a Senior Architect with Cisco Systems. He has been involved
with integrated management of networked systems and services since 1990. Alex has provided
technical leadership for many network management development and engineering efforts from
original conception to delivery to the customer. They include management instrumentation of
network devices, turnkey management solutions for packet telephony and managed services, and
management systems for Voice over IP networks, broadband access networks, and provisioning of
residential subscriber services. Alex has approximately 30 publications related to network
management and 15 patents pending. He is on the Organizing Committee or Technical Program
Committee of the major technical conferences in the field, including IM, NOMS, DSOM, IPOM,
and MMNS, and he served as Technical Program Co-chair of the 2005 IFIP/IEEE International
Symposium on Integrated Network Management. He holds a Ph.D. degree from the University of
Munich and a Master’s degree from Stanford University.


v

About the Technical Reviewers
Prakash Bettadapur is a Senior Engineering Manager at Cisco Systems. He has been with Cisco

since 1999, working in various network management and IOS manageability programs. Before
Cisco, Prakash worked in Bell Northern Research (BNR) in Ottawa, Canada, and in Nortel
Networks in Santa Clara, California, for 14 years. While in BNR/Nortel, Prakash worked in DMS–
Service Control Point, Data Packet Networking (DPN), Magellan Passport, and Meridian PBX
product lines, focusing on the areas of software development and network management. Prakash
holds a Master’s degree in computing science from the University of Alberta, Canada; a
Proficience Certificate in computing systems from the Indian Institute of Science, Bangalore; and
a Bachelor’s degree in electronics and telecommunications engineering from Karnataka Regional
Engineering College, India. Prakash currently lives in San Jose, California.
David M. Kurtiak is a Principal Engineer for Loral Skynet, where he currently architects systems
and network infrastructure and provides tier 3 support for the company’s global IT organization.
In a previous role at Skynet, Dave led a team of technical professionals responsible for managing
the daily operations of the company’s IT and data network infrastructure. Before joining Loral,
Dave was a senior data communications specialist for AT&T. David has more than 18 years of
experience in the IT and telecommunications industry, working in many telecommunications
technologies. He is recognized as the resident expert in TCP/IP networking, with specialization in
end-to-end network analysis, planning, troubleshooting, and performance tuning. David has a
Master’s degree (M.S.) in telecommunications from the University of Colorado at Boulder and a
Bachelor’s degree (B.S.) in information systems from the University of North Carolina at
Greensboro.
Lundy Lewis is the Chair of the Department of Information Technology at Southern New
Hampshire University. He has worked in the area of network management since the early 1990s.
He holds 22 U.S. patents and has written three books on network and service management. He is
a member of the technical committees for the major IEEE conferences on network management.


vi

Dedications
To my wonderful wife and kids—Sigrid, Clarissa, and Christopher. Thank you for

making me complete.


vii

Acknowledgments
At various stages of writing this book, I had interesting discussions, support, and valuable
feedback from many friends and colleagues. In particular, I would like to acknowledge Ron Biell,
Steve Chang, Eva Krüger, Victor Lee, Dave McNamee, Fred Schindler, Hector Trevino, Eshwar
Yedavalli, and Ralf Wolter. A very special “thank you” goes out to my dad, Helmut Clemm, who,
in fact, read through the entire manuscript and, although not a “network manager,” provided many
useful insights.
I also want to acknowledge this book’s production team, which is the finest anyone could ask for.
Specifically, I would like to acknowledge the people I interacted with the most—Jim Schachterle,
who first got the ball rolling; Raina Han and Mary Beth Ray, who accompanied me through most
of the writing stage; and Betsey Henkels, whose development edits were of great help during the
“crunch time” of the book; and Tonya Simpson, my project editor. The team also includes my
technical editors, Prakash Bettadapur, David Kurtiak, and Lundy Lewis, whose excellent
comments and suggestions undoubtedly helped to significantly improve the book.
Last but not least, I would like to thank my family for their understanding and support throughout
this project, which, by the nature of things, meant sacrificing many weekends; nonetheless, they
never stopped cheering me on. We did it!


viii

This Book Is Safari Enabled
The Safari® Enabled icon on the cover of your favorite technology book means
the book is available through Safari Bookshelf. When you buy this book, you get
free access to the online edition for 45 days.

Safari Bookshelf is an electronic reference library that lets you easily search
thousands of technical books, find code samples, download chapters, and access
technical information whenever and wherever you need it.
To gain 45-day Safari Enabled access to this book:
• Go to />
If you have difficulty registering on Safari Bookshelf or accessing the online
edition, please e-mail


ix

Contents at a Glance
Introduction

Part I

xix

Network Management: An Overview 3

Chapter 1

Setting the Stage

Chapter 2

On the Job with a Network Manager

Chapter 3

The Basic Ingredients of Network Management

Part II

5
47
75

Management Perspectives 101

Chapter 4

The Dimensions of Management

103

Chapter 5

Management Functions and Reference Models: Getting Organized

Part III Management Building Blocks 169

Chapter 6

Management Information: What Management Conversations Are
All About 171

Chapter 7

Management Communication Patterns: Rules of Conversation

209

Chapter 8

Common Management Protocols: Languages of Management

249

Chapter 9

Management Organization: Dividing the Labor

293

Part IV Applied Network Management 329

Chapter 10

Management Integration: Putting the Pieces Together

Chapter 11

Service Level Management: Knowing What You Pay For

Chapter 12

Management Metrics: Assessing Management Impact and
Effectiveness 407

Part V Appendixes 433

Appendix A

Answers to Chapter Reviews

Appendix B

Further Reading

Glossary
Index

475

488

463

435

331
373

129


x

Contents

Introduction

Part I

xix

Network Management: An Overview 3

Chapter 1

Setting the Stage

5

Defining Network Management 5
Analogy 1: Health Care—the Network, Your Number One Patient 6
Analogy 2: Throwing a Party 7
A More Formal Definition 8
The Importance of Network Management: Many Reasons to Care 10
Cost 12
Quality 14
Revenue 15
The Players: Different Parties with an Interest in Network Management 16
Network Management Users 16
The Service Provider 16
The Enterprise IT Department 17
The End User 18
Network Management Providers 19
The Equipment Vendor 19
The Third-Party Application Vendor 20

The Systems Integrator 20
Network Management Complexities: From Afterthought
to Key Topic 21
Technical Challenges 22
Application Characteristics 23
Scale 26
Cross-Section of Technologies 30
Integration 34
Organization and Operations Challenges 36
Functional Division of Tasks 37
Geographical Distribution 38
Operational Procedures and Contingency Planning 38
Business Challenges 39
Placing a Value on Network Management 40
Feature vs. Product 41
Uneven Competitive Landscape 42
Chapter Summary 44
Chapter Review 45

Chapter 2

On the Job with a Network Manager

47

A Day in the Life of a Network Manager 48
Pat: A Network Operator for a Global Service Provider 48
Chris: Network Administrator for a Medium-Size Business 54

xi

Sandy: Administrator and Planner in an Internet Data Center
Observations 62
The Network Operator’s Arsenal: Management Tools 63
Device Managers and Craft Terminals 64
Network Analyzers 65
Element Managers 65
Management Platforms 66
Collectors and Probes 67
Intrusion Detection Systems 67
Performance Analysis Systems 68
Alarm Management Systems 68
Trouble Ticket Systems 69
Work Order Systems 69
Workflow Management Systems and Workflow Engines 70
Inventory Systems 70
Service Provisioning Systems 71
Service Order–Management Systems 71
Billing Systems 72
Chapter Summary 72
Chapter Review 73

Chapter 3

The Basic Ingredients of Network Management

60

75

The Network Device 76
Management Agent 77
Management Information, MOs, MIBs, and Real Resources 80
Basic Management Ingredients—Revisited 83
The Management System 83
Management System and Manager Role 84
A Management System’s Reason for Being 86
The Management Network 86
Networking for Management 87
The Pros and Cons of a Dedicated Management Network 90
The Management Support Organization: NOC, NOC, Who’s There? 93
Managing the Management 93
Inside the Network Operations Center 96
Chapter Summary 97
Chapter Review 98

Part II

Management Perspectives 101

Chapter 4

The Dimensions of Management

103

Lost in (Management) Space: Charting Your Course Along Network Management
Dimensions 104
Management Interoperability: “Roger That” 104

Communication Viewpoint: Can You Hear Me Now? 106
Function Viewpoint: What Can I Do for You Today? 108


xii

Information Viewpoint: What Are You Talking About? 110
The Role of Standards 111
Management Subject: What We’re Managing 114
Management Life Cycle: Managing Networks from Cradle
to Grave 115
Planning 116
Deployment 117
Operations 117
Decommissioning 118
Management Layer: It’s a Device… No, It’s a Service… No, It’s a Business 118
Element Managment 119
Network Management 119
Service Management 120
Business Management 121
Network Element 121
Additional Considerations 121
Management Function: What’s in Your Toolbox 122
Management Process and Organization: Of Help Desks and Cookie Cutters 123
Chapter Summary 126
Chapter Review 127

Chapter 5

Management Functions and Reference Models: Getting Organized

Of Pyramids and Layered Cakes 129
FCAPS: The ABCs of Management 131
F Is for Fault 132
Network Monitoring Overview 132
Basic Alarm Management Functions 133
Advanced Alarm Management Functions 135
Alarm and Event Filtering 138
Alarm and Event Correlation 140
Fault Diagnosis and Troubleshooting 141
Proactive Fault Management 143
Trouble Ticketing 143
C Is for Configuration 143
Configuring Managed Resources 145
Auditing, Discovery, and Autodiscovery 146
Synchronization 148
Backup and Restore 151
Image Management 151
A Is for Accounting 151
On the Difference Between Billing and Accounting 152
Accounting for Communication Service Consumption 153
Accounting Management as a Service Feature 154
P Is for Performance 155
Performance Metrics 155

129


xiii

Monitoring and Tuning Your Network for Performance 156

Collecting Performance Data 157
S Is for Security 158
Security of Management 158
Management of Security 159
Limitations of the FCAPS Categorization 161
OAM&P: The Other FCAPS 161
FAB and eTOM: Oh, Wait, There’s More 163
How It All Relates and What It Means to You: Using Your Network Management ABCs
Chapter Summary 165
Chapter Review 166

Part III Management Building Blocks 169

Chapter 6

Management Information: What Management Conversations Are
All About 171
Establishing a Common Terminology Between Manager
and Agent 171
MIBs 173
The Managed Device as a Conceptual Data Store 173
Categories of Management Information 175
The Difference Between a MIB and a Database 177
The Relationship Between MIBs and Management Protocols 178
MIB Definitions 180
Of Schema and Metaschema 181
The Impact of the Metaschema on the Schema 183
Metaschema Modeling Paradigms 184
Matching Management Information and Metaschema 185
A Simple Modeling Example 186

Encoding Management Information 189
Anatomy of a MIB 189
Structure of Management Information—Overview 190
An Example: MIB-2 193
Instantiation in an Actual MIB 199
Special MIB Considerations to Address SNMP Protocol Deficits 202
Modeling Management Information 202
Chapter Summary 205
Chapter Review 206

Chapter 7

Management Communication Patterns: Rules of Conversation
Layers of Management Interactions
Transport 211
Remote Operations 211
Management Operations 214
Management Services 215

209

209

164


xiv

Manager-Initiated Interactions—Request and Response 216
Information Retrieval—Polling and Polling-Based Management 218

Requests for Configuration Information 218
Requests for Operational Data and State Information 219
Bulk Requests and Incremental Operations 223
Historical Information 224
Configuration Operations 226
Failure Recovery 227
Response Size and Request Scoping 228
Dealing with Configuration Files 229
Actions 230
Management Transactions 232
Agent-Initiated Interactions: Events and Event-Based Management 236
Event Taxonomy 237
Alarms 238
Configuration-Change Events 239
Threshold-Crossing Alerts 241
The Case for Event-Based Management 243
Reliable Events 244
On the Difference Between “Management” and “Control” 245
Chapter Summary 246
Chapter Review 247

Chapter 8

Common Management Protocols: Languages of Management
SNMP: Classic and Perennial Favorite 249
SNMP “Classic,” a.k.a. SNMPv1 250
SNMP Operations 250
SNMP Messages and Message Structure 257
SNMPv2/ SNMPv2c 258
SNMPv3 260

CLI: Management Protocol of Broken Dreams 261
CLI Overview 261
Use of CLI as a Management Protocol 265
syslog: The CLI Notification Sidekick 267
syslog Overview 268
syslog Protocol 270
syslog Deployment 272
Netconf: A Management Protocol for a New Generation 275
Netconf Datastores 275
Netconf and XML 277
Netconf Architecture 278
Netconf Operations 281
Netflow and IPFIX: “Check, Please,” or, All the Data, All the Time
IP Flows 284
Netflow Protocol 286

284

249


xv

Chapter Summary 288
Chapter Review 291

Chapter 9

Management Organization: Dividing the Labor

293

Scaling Network Management 294
Management Complexity 294
Build Complexity 295
Runtime Complexity 297
Management Hierarchies 298
Subcontracting Management Tasks 299
Deployment Aspects 301
Management Styles 304
Management by Delegation 304
Management by Objectives and Policy-Based Management 308
Management by Exception 312
Management Mediation 312
Mediation Between Management Transports 316
Mediation Between Management Protocols 316
Mediation of Management Information at the Syntactic Level 318
Example: A Syslog-to-SNMP Management Gateway 318
Example: An SNMP-to-OO Management Gateway 319
Limitations of Syntactic Information Mediation 321
Mediation of Management Information at the Semantic Level 323
Stateful Mediation 323
Chapter Summary 326
Chapter Review 327

Part IV Applied Network Management 329

Chapter 10

Management Integration: Putting the Pieces Together

331

The Need for Management Integration 332
Benefits of Integrated Management 332
Nontechnical Considerations for Management Integration 334
Different Perspectives on Management Integration Needs 336
The Equipment Vendor Perspective 336
The Enterprise Perspective 338
The Service Provider Perspective 339
Integration Scope and Complexity 340
Management Integration Challenges 342
Managed Domain 343
Software Architecture 345
Challenges from Application Requirements 345
Challenges from Conflicting Software Architecture Goals 346
Eierlegende Wollmilchsaun and One-Size-Fits-All Management Systems
Quantifying Management Integration Complexity 348
Scale Complexity 349

348


xvi

Heterogeneity Complexity 349
Function Complexity 350
Approaches to Management Integration 351
Adapting Integration Approach and Network Provider Organization 352
Platform Approach 355

Common Platform Infrastructure 356
Typical Platform Application Functionality 359
Custom Integration Approach 360
Solution Philosophy and Challenges 360
Considerations for Top-Down Solution Design 362
Component Integration Levels and Bottom-Up Solution Design 365
The Role of Standardization and Information Models 367
Containing Complexity of the Managed Domain 368
Chapter Summary 370
Chapter Review 371

Chapter 11

Service Level Management: Knowing What You Pay For

373

The Motivation for Service Level Agreements 374
Identification of Service Level Parameters 376
Significance 377
A Brief Detour: Service Level Relationships Between Layered Communication
Services 377
Example: Voice Service Level Parameters 379
Relevance 381
Measurability 381
Defining a Service Level Agreement 382
Definition of Service Level Objectives 382
Tracking Service Level Objectives 384
Dealing with Service Level Violations 386
Managing for a Service Level 388

Decomposing Service Level Parameters 389
Planning Networks for a Given Service Level 392
Dimensioning Networks to Meet Service Level Objectives 393
Managing Oversubscription Risk 394
Network Maintenance Considerations 396
Service Level Monitoring—Setting Up Early Warning Systems 397
Monitoring Service Level Parameters 397
Anticipating Problems Before They Occur 398
Service Level Statistics—It’s Fingerpointin’ Good 400
Chapter Summary 402
Chapter Review 403


xvii

Chapter 12

Management Metrics: Assessing Management Impact and
Effectiveness 407
Network Management Business Impact 408
Cost of Ownership 408
Enabling of Revenues 409
Network Availability 410
Trading Off the Benefits and Costs of Network Management Investments 410
Factors that Determine Management Effectiveness 411
Managed Technology—Manageability 412
Management Systems and Operations Support Infrastructure 416
Management Organization 418
Assessing Network Management Effectiveness 418
Management Metrics to Track Business Impact 419

Management Metrics to Track Contribution to Management Effectiveness 423
Metrics for Complexity of Operational Tasks 423
Metrics for Scale 425
Other Metrics 426
Developing Your Own Management Benchmark 427
Assessing and Tracking the State of Management 428
Using Metrics to Direct Management Investment 430
Chapter Summary 430
Chapter Review 431

Part V Appendixes 433

Appendix A

Answers to Chapter Reviews

Appendix B

Further Reading

Glossary
Index

475

488

463

435

xviii

Icons Used in This Book

Communication
Server

PC

PC with
Software

Terminal

File
Server

Sun
Workstation

Macintosh

Access
Server

ISDN/Frame Relay
Switch

Ciscoworks
Workstation

ATM
Switch

Modem

Token
Ring
Token Ring

Printer

Laptop

Web
Server

IBM
Mainframe

Front End
Processor

Cluster
Controller

Multilayer
Switch

FDDI
Gateway

Router

Network Cloud

Bridge

Line: Ethernet

Hub

DSU/CSU
DSU/CSU

Line: Serial

FDDI

Catalyst
Switch

Line: Switched Serial

Command Syntax Conventions
The conventions used to present command syntax in this book are the same conventions used in
the IOS Command Reference. The Command Reference describes these conventions as follows:
■

Boldface indicates commands and keywords that are entered literally as shown. In actual
configuration examples and output (not general command syntax), boldface indicates
commands that are manually input by the user (such as a show command).

■

Italics indicate arguments for which you supply actual values.

■

Vertical bars (|) separate alternative, mutually exclusive elements.

■

Square brackets ([ ]) indicate optional elements.

■

Braces ({ }) indicate a required choice.

■

Braces within brackets ([{ }]) indicate a required choice within an optional element.


xix

Introduction
Network management is an essential factor in successfully operating a network. As businesses

become increasingly dependent on networking services, keeping those services running becomes
synonymous with keeping the business running.
Properly performed, network management ensures that services provided over a network are
turned up swiftly and keep running smoothly. In addition, network management helps to keep
networking cost and operational cost under control. It ensures that networking equipment is used
effectively and deployed where it is needed the most. It increases the availability and quality of
the services that the network provides. At least in the case of service providers, it is also a
significant factor in the generation of revenue from networking services. On the other hand,
ineffective management can lead to deterioration and disruption of networking services, poor
utilization of investment made in the network, and lost business. Network management is hence
key to getting the most value out of a network and can be absolutely business critical.
Despite its significance, network management is without much doubt one of the lesser understood
topics in the otherwise well-charted world of networking. Reasons for this include the fact that
network management looks deceptively simple, whereas it can be difficult to master, and that it is
overshadowed by the networking technology itself that it is supposed to manage.
In some ways, managing a network is like throwing a party: Most people enjoy going to a party
(read: the services provided by the network) but do not want to deal with the hassle of setting it
up, keeping everything flowing smoothly, and cleaning up the mess afterward (read: network
management). Yet this is essential to the party’s success (and ensuring that there will be another
one). As with network management, many technical disciplines are involved: Food needs to be
cooked, rooms decorated, invitations printed, and electrical equipment and lighting set up. And as
with network management, organizational and business questions abound: Do I throw it at my
home, or do I lease a location? Where will I put the coats? How many drinks do I need? Can I do
it all by myself, or at what point does it make sense to use a caterer?
Network Management Fundamentals aims to provide an accessible introduction to this important
subject area. It covers management not just of networks themselves, but also of services running
over those networks. It explains the fundamental concepts and principles that network
management is based on. It attempts to provide a holistic system perspective of network
management and explains how different technologies that are used in network management relate
to each other. This system perspective aims to convey a sense of the forest rather than of the

individual trees. Hopefully, the resulting understanding will put you, the reader, in a position in
which you can successfully navigate the subject area of network management and apply its
concepts to your particular situation.


xx

Who Should Read This Book?
This book is intended as an introduction and guide to network management for anyone interested
in the topic, whether that person has only a basic understanding of networking technology and is
only casually interested in the subject, or whether that person is an experienced networking
professional looking to expand his or her core competencies. The book tries to avoid overloading
the reader with unnecessary complexity and details that would distract from these fundamentals
and key concepts, yet provide a solid technical foundation for the practitioner.
The target audience includes network operators, development engineers, test engineers, operations
planners, project managers, and product managers who need to deal with network management in
some way as part of their jobs. It also includes executives who need to understand the impact of
network management on their organization, as well as engineering students who want to round off
a networking curriculum.
The emphasis in this book lies on fundamentals and general principles in network management
rather than technical details and “how-to” instructions. Accordingly, if you are interested in the
details of a particular management protocol or in the specifics of a particular management
application, this is not the right book for you. If, on the other hand, you want to understand the
foundations of network management and how management technology really works, this book
should prove useful to you.