hack the stack - using snort & ethereal to master the 8 layers of an insecure network

www.syngress.com
Syngress is committed to publishing high-quality books for IT Professionals and
delivering those books in media and formats that fit the demands of our cus-
tomers. We are also committed to extending the utility of the book you purchase
via additional materials available from our Web site.
SOLUTIONS WEB SITE
To register your book, visit www.syngress.com/solutions. Once registered, you can
access our Web pages. There you will find an assortment
of value-added features such as free e-books related to the topic of this book, URLs
of related Web site, FAQs from the book, corrections, and any updates from the
author(s).
ULTIMATE CDs
Our Ultimate CD product line offers our readers budget-conscious compilations of
some of our best-selling backlist titles in Adobe PDF form. These CDs are the perfect
way to extend your reference library on key topics pertaining to your area of exper-
tise, including Cisco Engineering, Microsoft Windows System Administration,
CyberCrime Investigation, Open Source Security, and Firewall Configuration, to
name a few.
DOWNLOADABLE E-BOOKS
For readers who can’t wait for hard copy, we offer most of our titles in download-
able Adobe PDF form. These eBooks are often available weeks before hard copies,
and are priced affordably.
SYNGRESS OUTLET
Our outlet store at syngress.com features overstocked, out-of-print, or slightly hurt
books at significant savings.
SITE LICENSING
Syngress has a well-established program for site licensing our e-books onto servers
in corporations, educational institutions, and large organizations. Contact us at
for more information.
CUSTOM PUBLISHING

Many organizations welcome the ability to combine parts of multiple Syngress
books, as well as their own content, into a single volume for their own internal use.
Contact us at for more information.
Visit us at
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page i
Hack
the Stack
USING SNORT AND ETHEREAL TO MASTER
THE 8 LAYERS OF AN INSECURE NETWORK
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page iii
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page iv
Michael Gregg
Stephen Watkins Technical Editor
George Mays
Chris Ries
Ron Bandes
Brandon Franklin
Hack
the Stack
USING SNORT AND ETHEREAL TO MASTER
THE 8 LAYERS OF AN INSECURE NETWORK
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page v
Syngress Publishing, Inc., the author(s), and any person or firm involved in the writing, editing, or produc-
tion (collectively “Makers”) of this book (“the Work”) do not guarantee or warrant the results to be
obtained from the Work.
There is no guarantee of any kind, expressed or implied, regarding the Work or its contents.The Work is
sold AS IS and WITHOUT WARRANTY.You may have other legal rights, which vary from state to
state.
In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other
incidental or consequential damages arising out from the Work or its contents. Because some states do not

allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation
may not apply to you.
You should always use reasonable care, including backup and other appropriate precautions, when working
with computers, networks, data, and files.
Syngress Media®, Syngress®,“Career Advancement Through Skill Enhancement®,” “Ask the Author
UPDATE®,” and “Hack Proofing®,” are registered trademarks of Syngress Publishing, Inc.“Syngress:The
Definition of a Serious Security Library”™, “Mission Critical™,” and “The Only Way to Stop a Hacker is
to Think Like One™” are trademarks of Syngress Publishing, Inc. Brands and product names mentioned
in this book are trademarks or service marks of their respective companies.
KEY SERIAL NUMBER
001 HJIRTCV764
002 PO9873D5FG
003 829KM8NJH2
004 HEATHTANER
005 CVPLQ6WQ23
006 VBP965T5T5
007 HJJJ863WD3E
008 2987GVTWMK
009 629MP5SDJT
010 IMWQ295T6T
PUBLISHED BY
Syngress Publishing, Inc.
800 Hingham Street
Rockland, MA 02370
Hack the Stack: Using Snort and Ethereal to Master the 8 Layers of an Insecure Netork
Copyright © 2006 by Syngress Publishing, Inc. All rights reserved. Printed in the United States of
America. Except as permitted under the Copyright Act of 1976, no part of this publication may be repro-
duced or distributed in any form or by any means, or stored in a database or retrieval system, without the
prior written permission of the publisher, with the exception that the program listings may be entered,
stored, and executed in a computer system, but they may not be reproduced for publication.

Printed in the Canada
1 2 3 4 5 6 7 8 9 0
ISBN: 1-59749-109-8
Publisher: Andrew Williams Page Layout and Art: Patricia Lupien
Acquisitions Editor: Gary Byrne Copy Editor: Judy Eby
Technical Editor: Stephen Watkins Indexer: Odessa&Cie
Cover Designer: Michael Kavish
Distributed by O’Reilly Media, Inc. in the United States and Canada.
For information on rights, translations, and bulk sales, contact Matt Pedersen, Director of Sales and Rights,
at Syngress Publishing; email or fax to 781-681-3585.
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page vi
Acknowledgments
vii
Syngress would like to acknowledge the following people for their kindness
and support in making this book possible.
Syngress books are now distributed in the United States and Canada by
O’Reilly Media, Inc.The enthusiasm and work ethic at O’Reilly are incredible,
and we would like to thank everyone there for their time and efforts to bring
Syngress books to market:Tim O’Reilly, Laura Baldwin, Mark Brokering, Mike
Leonard, Donna Selenko, Bonnie Sheehan, Cindy Davis, Grant Kikkert, Opol
Matsutaro, Steve Hazelwood, Mark Wilson, Rick Brown,Tim Hinton, Kyle
Hart, Sara Winge, Peter Pardo, Leslie Crandell, Regina Aggio Wilkinson, Pascal
Honscher, Preston Paull, Susan Thompson, Bruce Stewart, Laura Schmier, Sue
Willing, Mark Jacobsen, Betsy Waliszewski, Kathryn Barrett, John Chodacki,
Rob Bullington, Kerry Beck, Karen Montgomery, and Patrick Dirden.
The incredibly hardworking team at Elsevier Science, including Jonathan
Bunkell, Ian Seager, Duncan Enright, David Burton, Rosanna Ramacciotti,
Robert Fairbrother, Miguel Sanchez, Klaus Beran, Emma Wyatt, Krista
Leppiko, Marcel Koppes, Judy Chappell, Radek Janousek, Rosie Moss, David
Lockley, Nicola Haden, Bill Kennedy, Martina Morris, Kai Wuerfl-Davidek,

Christiane Leipersberger,Yvonne Grueneklee, Nadia Balavoine, and Chris
Reinders for making certain that our vision remains worldwide in scope.
David Buckland, Marie Chieng, Lucy Chong, Leslie Lim, Audrey Gan, Pang Ai
Hua, Joseph Chan, June Lim, and Siti Zuraidah Ahmad of Pansing Distributors
for the enthusiasm with which they receive our books.
David Scott, Tricia Wilden, Marilla Burgess, Annette Scott, Andrew Swaffer,
Stephen O’Donoghue, Bec Lowe, Mark Langley, and Anyo Geddes of Woodslane
for distributing our books throughout Australia, New Zealand, Papua New
Guinea, Fiji,Tonga, Solomon Islands, and the Cook Islands.
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page vii
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page viii
ix
Lead Author
Michael Gregg is the President of Superior Solutions, Inc.
and has more than 20 years’ experience in the IT field. He
holds two associate’s degrees, a bachelor’s degree, and a master’s
degree and is certified as CISSP, MCSE, MCT, CTT+, A+,
N+, Security+, CNA, CCNA, CIW Security Analyst, CCE,
CEH, CHFI, CEI, DCNP, ES Dragon IDS, ES Advanced
Dragon IDS, and TICSA.
Michael’s primary duties are to serve as project lead for
security assessments helping businesses and state agencies
secure their IT resources and assets. Michael has authored four
books, including Inside Network Security Assessment, CISSP Prep
Questions, CISSP Exam Cram2, and Certified Ethical Hacker
Exam Prep2. He has developed four high-level security classes,
including Global Knowledge’s Advanced Security Boot Camp,
Intense School’s Professional Hacking Lab Guide,ASPE’s
Network Security Essentials, and Assessing Network
Vulnerabilities. He has created over 50 articles featured in mag-

azines and Web sites, including Certification Magazine,
GoCertify, The El Paso Times, and SearchSecurity.
Michael is also a faculty member of Villanova University
and creator of Villanova’s college-level security classes,
including Essentials of IS Security, Mastering IS Security, and
Advanced Security Management. He also serves as a site expert
for four TechTarget sites, including SearchNetworking,
SearchSecurity, SearchMobileNetworking, and SearchSmallBiz.
He is a member of the TechTarget Editorial Board.
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page ix
xx
Ronald T. Bandes (CISSP, CCNA, MCSE, Security+) is an
independent security consultant. Before becoming an indepen-
dent consultant, he performed security duties for Fortune 100
companies such as JP Morgan, Dun and Bradstreet, and EDS.
Ron holds a B.A. in Computer Science.
Brandon Franklin (GCIA, MCSA, Security+) is a network
administrator with KIT Solutions. KIT Solutions, Inc. (KIT
stands for Knowledge Based Information Technology) creates
intelligent systems for the health and human services industry
that monitor and measure impact and performance outcomes
and provides knowledge for improved decision making. A KIT
system enables policy makers, government agencies, private
foundations, researchers, and field practitioners to implement
best practices and science-based programs, demonstrate
impacts, and continuously improve outcomes.
Brandon formerly served as the Team Lead of Intrusion
Analysis at VigilantMinds, a Pittsburgh-based managed security
services provider.
Brandon cowrote Chapter 3 and wrote Chapter 6.

Contributing Authors
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page x
xi
George Mays (CISSP, CCNA,A+, Network+, Security+, I-
Net+) is an independent consultant who has 35 years’ experi-
ence in computing, data communications, and network
security. He holds a B.S. in Systems Analysis. He is a member
of the IEEE, CompTIA, and Internet Society.
Chris Ries is a Security Research Engineer for VigilantMinds
Inc., a managed security services provider and professional
consulting organization based in Pittsburgh. His research
focuses on the discovery, exploitation, and remediation of soft-
ware vulnerabilities, analysis of malicious code, and evaluation
of security software. Chris has published a number of advi-
sories and technical whitepapers based on his research and has
contributed to several books on information security.
Chris holds a bachelor’s degree in Computer Science with
a Mathematics Minor from Colby College, where he com-
pleted research involving automated malicious code detection.
Chris has also worked as an analyst at the National Cyber-
Forensics & Training Alliance (NCFTA) where he conducted
technical research to support law enforcement.
Chris wrote Chapter 8.
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page xi
xiixii
Stephen Watkins (CISSP) is an Information Security
Professional with more than 10 years of relevant technology
experience, devoting eight of these years to the security field.
He currently serves as Information Assurance Analyst at
Regent University in southeastern Virginia. Before coming to

Regent, he led a team of security professionals providing in-
depth analysis for a global-scale government network. Over the
last eight years, he has cultivated his expertise with regard to
perimeter security and multilevel security architecture. His
Check Point experience dates back to 1998 with FireWall-1
version 3.0b. He has earned his B.S. in Computer Science from
Old Dominion University and M.S. in Computer Science,
with Concentration in Infosec, from James Madison University.
He is nearly a life-long resident of Virginia Beach, where he
and his family remain active in their Church and the local
Little League.
Stephen wrote Chapter 7.
Technical Editor
408_Hack_the_Stack_FM.qxd 9/22/06 7:44 PM Page xii
Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
Chapter 1 Extending OSI to Network Security . . . . . . . . 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Our Approach to This Book . . . . . . . . . . . . . . . . . . . . . . . . .2
Tools of the Trade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Protocol Analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . .3
Organization of This Book . . . . . . . . . . . . . . . . . . . . . . .4
The People Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
The Application Layer . . . . . . . . . . . . . . . . . . . . . . . .6
The Presentation Layer . . . . . . . . . . . . . . . . . . . . . . . .6
The Session Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
The Transport Layer . . . . . . . . . . . . . . . . . . . . . . . . .6
The Network Layer . . . . . . . . . . . . . . . . . . . . . . . . . .7
The Data Link Layer . . . . . . . . . . . . . . . . . . . . . . . . .7
The Physical Layer . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Common Stack Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
The People Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
The Application Layer . . . . . . . . . . . . . . . . . . . . . . . . . . .8
The Session Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
The Transport Layer . . . . . . . . . . . . . . . . . . . . . . . . . . .10
The Data Link Layer . . . . . . . . . . . . . . . . . . . . . . . . . .11
The Physical Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Mapping OSI to TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . .13
Countermeasures Found in Each Layer . . . . . . . . . . . . .14
The Current State of IT Security . . . . . . . . . . . . . . . . . . . .16
Physical Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Communications Security . . . . . . . . . . . . . . . . . . . . . . .17
xiii
Contents
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xiii
xiv Contents
Signal Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Computer Security . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Network Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Information Security . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Using the Information in This Book . . . . . . . . . . . . . . . . . .19
Vulnerability Testing . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Security Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Finding and Reporting Vulnerabilities . . . . . . . . . . . . . .21
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . .25
Chapter 2 The Physical Layer . . . . . . . . . . . . . . . . . . . . . 27
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Defending the Physical Layer . . . . . . . . . . . . . . . . . . . . . . . .28

Design Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Perimeter Security . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Fencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Gates, Guards, and Grounds Design . . . . . . . . . . . . . .32
Facility Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Entry Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Device Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Identification and Authentication . . . . . . . . . . . . . . .39
Computer Controls . . . . . . . . . . . . . . . . . . . . . . . . .41
Mobile Devices and Media . . . . . . . . . . . . . . . . . . . .41
Communications Security . . . . . . . . . . . . . . . . . . . . . . .44
Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
802.11 Wireless Protocols . . . . . . . . . . . . . . . . . . . . .46
Attacking the Physical Layer . . . . . . . . . . . . . . . . . . . . . . . .47
Stealing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Data Slurping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Lock Picks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Wiretapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Scanning and Sniffing . . . . . . . . . . . . . . . . . . . . . . . . . .54
The Early History of Scanning and Sniffing . . . . . . . .54
Modern Wireless Vulnerabilities . . . . . . . . . . . . . . . .55
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xiv
Contents xv
Hardware Hacking . . . . . . . . . . . . . . . . . . . . . . . . . . . .57
Bypassing Physical Controls . . . . . . . . . . . . . . . . . . .58
Modifying Hardware . . . . . . . . . . . . . . . . . . . . . . . .59
Layer 1 Security Project . . . . . . . . . . . . . . . . . . . . . . . . . . .64
One-Way Data Cable . . . . . . . . . . . . . . . . . . . . . . . . . .64
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . .67
Chapter 3 Layer 2: The Data Link Layer. . . . . . . . . . . . . 69
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70
Ethernet and the Data Link Layer . . . . . . . . . . . . . . . . . . . .70
The Ethernet Frame Structure . . . . . . . . . . . . . . . . . . . .71
Understanding MAC Addressing . . . . . . . . . . . . . . . . . .72
Identifying Vendor Information . . . . . . . . . . . . . . . . .72
Performing Broadcast and Multicast . . . . . . . . . . . . .73
Examining the EtherType . . . . . . . . . . . . . . . . . . . . . . .73
Understanding PPP and SLIP . . . . . . . . . . . . . . . . . . . . . . .73
Examining SLIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Examining PPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74
Working with a Protocol Analyzer . . . . . . . . . . . . . . . . . . . .75
Writing BPFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77
Examining Live Traffic . . . . . . . . . . . . . . . . . . . . . . . . . .78
Filtering Traffic, Part Two . . . . . . . . . . . . . . . . . . . . . . . .79
Understanding How ARP Works . . . . . . . . . . . . . . . . . . . .82
Examining ARP Packet Structure . . . . . . . . . . . . . . . . .82
Attacking the Data Link Layer . . . . . . . . . . . . . . . . . . . . . . .84
Passive versus Active Sniffing . . . . . . . . . . . . . . . . . . . . .85
ARP Poisoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85
ARP Flooding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Routing Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Sniffing Wireless . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Netstumbler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Kismet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Cracking WEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89
Wireless Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . .90
Conducting Active Wireless Attacks . . . . . . . . . . . . . .90

Jamming Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xv
xvi Contents
MITM Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Defending the Data Link Layer . . . . . . . . . . . . . . . . . . . . . .91
Securing Your Network from Sniffers . . . . . . . . . . . . . . . . .91
Using Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Secure Shell (SSH) . . . . . . . . . . . . . . . . . . . . . . . . . .92
Secure Sockets Layers (SSL) . . . . . . . . . . . . . . . . . . . . .92
PGP and S/MIME . . . . . . . . . . . . . . . . . . . . . . . . . . . .92
Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
Employing Detection Techniques . . . . . . . . . . . . . . . . . . . . .93
Local Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
Network Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
DNS Lookups . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
Driver Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
Network Monitor . . . . . . . . . . . . . . . . . . . . . . . . . .95
Using Honeytokens . . . . . . . . . . . . . . . . . . . . . . . . .95
Data Link Layer Security Project . . . . . . . . . . . . . . . . . . . . .95
Using the Auditor Security Collection to Crack WEP . . . . .95
Cracking WEP with the Aircrack Suite . . . . . . . . . . .96
Cracking WPA with CoWPAtty . . . . . . . . . . . . . . . .98
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .101
Chapter 4 Layer 3: The Network Layer . . . . . . . . . . . . 103
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104
The IP Packet Structure . . . . . . . . . . . . . . . . . . . . . . . . . .104
Identifying IP’s Version . . . . . . . . . . . . . . . . . . . . . . . .106

Type of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107
Total Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110
Datagram ID Number . . . . . . . . . . . . . . . . . . . . . . . . .110
Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111
Time to Live (TTL) . . . . . . . . . . . . . . . . . . . . . . . . . .112
Protocol Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
Checksum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
IP Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xvi
Contents xvii
The ICMP Packet Structure . . . . . . . . . . . . . . . . . . . . . . .118
ICMP Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118
ICMP Message Types and Format . . . . . . . . . . . . . . . .118
Common ICMP Messages . . . . . . . . . . . . . . . . . . . . . .119
Destination Unreachable . . . . . . . . . . . . . . . . . . . . .120
Traceroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121
Path MTU Discovery . . . . . . . . . . . . . . . . . . . . . .122
Redirects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .122
Attacking the Network Layer . . . . . . . . . . . . . . . . . . . . . .123
IP Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124
Spoofing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124
Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . .124
Passive Fingerprinting . . . . . . . . . . . . . . . . . . . . . . .126
p0f—a Passive Fingerprinting Tool . . . . . . . . . . . . . .129
IP’s Role in Port Scanning . . . . . . . . . . . . . . . . . . .131
ICMP Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133
Covert Channels . . . . . . . . . . . . . . . . . . . . . . . . . . .133
ICMP Echo Attacks . . . . . . . . . . . . . . . . . . . . . . . .136
Port Scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . .136

OS Fingerprinting . . . . . . . . . . . . . . . . . . . . . . . . .137
DoS Attacks and Redirects . . . . . . . . . . . . . . . . . . .137
Router and Routing Attacks . . . . . . . . . . . . . . . . . . . .138
Network Spoofing . . . . . . . . . . . . . . . . . . . . . . . . .139
Defending the Network Layer . . . . . . . . . . . . . . . . . . . . . .140
Securing IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140
Securing ICMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140
Securing Routers and Routing Protocols . . . . . . . . . . .141
Address Spoofing . . . . . . . . . . . . . . . . . . . . . . . . . .142
Network Layer Security Project . . . . . . . . . . . . . . . . . . . . .143
Ptunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143
ACKCMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .149
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xvii
xviii Contents
Chapter 5 Layer 4: The Transport Layer. . . . . . . . . . . . 151
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152
Connection-Oriented versus Connectionless Protocols . . . .152
Connection-Oriented Protocols . . . . . . . . . . . . . . . . . .152
Connectionless Protocols . . . . . . . . . . . . . . . . . . . . . . .153
Why Have Both Kinds of Protocols? . . . . . . . . . . . . . .153
Protocols at the Transport Layer . . . . . . . . . . . . . . . . . . . . .153
UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154
TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .155
Source and Destination Ports . . . . . . . . . . . . . . . . . .156
Source Sequence Number
and Acknowledgment Sequence Number . . . . . . . .157
Data Offset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158

Control Bits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158
Window Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
Checksum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
Urgent Pointer . . . . . . . . . . . . . . . . . . . . . . . . . . . .160
How TCP Sessions Begin and End . . . . . . . . . . . . . . . .160
TCP Session Startup . . . . . . . . . . . . . . . . . . . . . . . .160
TCP Session Teardown . . . . . . . . . . . . . . . . . . . . . .161
The Hacker’s Perspective . . . . . . . . . . . . . . . . . . . . . . . . . .162
Some Common Attacks . . . . . . . . . . . . . . . . . . . . . . . .163
Scanning the Network . . . . . . . . . . . . . . . . . . . . . . . . . . .163
Port Scanning Overview . . . . . . . . . . . . . . . . . . . . . . .164
TCP Scan Variations . . . . . . . . . . . . . . . . . . . . . . . . . .165
Nmap Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .165
Nmap:The Most Well Known Scanning Tool . . . . . .167
Amap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170
Scanrand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .172
Operating System Fingerprinting . . . . . . . . . . . . . . . . . . . .173
How OS Discovery Works . . . . . . . . . . . . . . . . . . . . . .174
Xprobe2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176
OS Fingerprinting with Nmap . . . . . . . . . . . . . . . . . .179
Detecting Scans on Your Network . . . . . . . . . . . . . . . . . . .181
Snort Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xviii
Contents xix
The Snort User Interface—
Basic Analysis and Security Engine . . . . . . . . . . . . .182
Defending the Transport Layer . . . . . . . . . . . . . . . . . . . . . .183
How the SSL Protocol Operates . . . . . . . . . . . . . . . . .184
Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .184
Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185

Phase 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185
How SSL Appears on the Network . . . . . . . . . . . . . . .185
SSL/TLS Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .187
Transport Layer Project—Setting Up Snort . . . . . . . . . . . .187
Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .188
Install Fedora Core 4 . . . . . . . . . . . . . . . . . . . . . . . . . .188
Install Supporting Software . . . . . . . . . . . . . . . . . . . . .190
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .200
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .200
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .202
Chapter 6 Layer 5: The Session Layer . . . . . . . . . . . . . 205
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206
Attacking the Session Layer . . . . . . . . . . . . . . . . . . . . . . . .206
Observing a SYN Attack . . . . . . . . . . . . . . . . . . . . . . .206
Session Hijacking . . . . . . . . . . . . . . . . . . . . . . . . . . . .209
Session Hijacking Tools . . . . . . . . . . . . . . . . . . . . . .213
Domain Name System (DNS) Poisoning . . . . . . . . .216
Sniffing the Session Startup . . . . . . . . . . . . . . . . . . . . .218
Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .219
Authenticating with Password
Authentication Protocol . . . . . . . . . . . . . . . . . . . . .219
Authenticating with the Challenge Handshake
Authentication Protocol . . . . . . . . . . . . . . . . . . . . .219
Authenticating with Local
Area Network Manager and NT LAN Manager . . .220
Authenticating with NTLMv2 . . . . . . . . . . . . . . . .220
Authenticating with Kerberos . . . . . . . . . . . . . . . . .220
Tools Used for Sniffing the Session Startup . . . . . . .221
Observing a RST Attack . . . . . . . . . . . . . . . . . . . . . . .223
Defeating Snort at the Session Layer . . . . . . . . . . . . . . .224

408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xix
xx Contents
Defending the Session Layer . . . . . . . . . . . . . . . . . . . . . . .227
Mitigating DoS Attacks . . . . . . . . . . . . . . . . . . . . . . . .227
Preventing Session Hijacking . . . . . . . . . . . . . . . . . . . .228
Selecting Authentication Protocols . . . . . . . . . . . . . . . .229
Defending Against RST Attacks . . . . . . . . . . . . . . . . . .231
Detecting Session Layer Attacks . . . . . . . . . . . . . . . . . .232
Port Knocking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232
Session Layer Security Project . . . . . . . . . . . . . . . . . . . . . .232
Using Snort to Detect Malicious Traffic . . . . . . . . . . . .233
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .237
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .237
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .239
Chapter 7 Layer 6: The Presentation Layer . . . . . . . . . 241
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .242
The Structure of NetBIOS and SMB . . . . . . . . . . . . . . . . .242
Attacking the Presentation Layer . . . . . . . . . . . . . . . . . . . .245
NetBIOS and Enumeration . . . . . . . . . . . . . . . . . . . . .245
Exploiting the IPC$ Share . . . . . . . . . . . . . . . . . . .247
Sniffing Encrypted Traffic . . . . . . . . . . . . . . . . . . . . . .250
Attacking Kerberos . . . . . . . . . . . . . . . . . . . . . . . . .253
Tools to Intercept Traffic . . . . . . . . . . . . . . . . . . . . .257
Defending the Presentation Layer . . . . . . . . . . . . . . . . . . .266
Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .266
The Role of IPSec . . . . . . . . . . . . . . . . . . . . . . . . . . .268
Protecting E-mail . . . . . . . . . . . . . . . . . . . . . . . . . . . .272
Secure/Multipurpose Internet Mail Extensions . . . .272
Tightening NetBIOS Protections . . . . . . . . . . . . . . . . .273
Presentation Layer Security Project . . . . . . . . . . . . . . . . . .274

Subverting Encryption and Authentication . . . . . . . . . .274
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .280
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .280
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .282
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .283
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xx
Contents xxi
Chapter 8 Layer 7: The Application Layer . . . . . . . . . . 285
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .286
The Structure of FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . .286
FTP Protocol Overview . . . . . . . . . . . . . . . . . . . . . . .286
FTP Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .288
FTP Security Issues . . . . . . . . . . . . . . . . . . . . . . . . . . .291
Analyzing Domain Name System and Its Weaknesses . . . . .292
DNS Message Format . . . . . . . . . . . . . . . . . . . . . . . . .292
The DNS Lookup Process . . . . . . . . . . . . . . . . . . . . . .295
The DNS Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . .296
Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .296
Zones and Zone Transfers . . . . . . . . . . . . . . . . . . . . . .297
DNS Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .297
DNS Security Issues . . . . . . . . . . . . . . . . . . . . . . . . . .298
Other Insecure Application Layer Protocols . . . . . . . . . . . .299
Simple Mail Transfer Protocol . . . . . . . . . . . . . . . . . . .299
SMTP Protocol Overview . . . . . . . . . . . . . . . . . . .299
SMTP Security Issues . . . . . . . . . . . . . . . . . . . . . . .300
Telnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .301
Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . . .302
Security Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . .302
Other Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .302
Attacking the Application Layer . . . . . . . . . . . . . . . . . . . . .303

Attacking Web Applications . . . . . . . . . . . . . . . . . . . . .303
SQL Injection . . . . . . . . . . . . . . . . . . . . . . . . . . . .303
Code Injection . . . . . . . . . . . . . . . . . . . . . . . . . . . .304
Cross-Site Scripting . . . . . . . . . . . . . . . . . . . . . . . .305
Directory Traversal Attacks . . . . . . . . . . . . . . . . . . .307
Information Disclosure . . . . . . . . . . . . . . . . . . . . . .307
Authentication and Access Control Vulnerabilities . .308
CGI Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . .308
Attacking DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . .308
Information Gathering . . . . . . . . . . . . . . . . . . . . . .309
DNS Cache Poisoning . . . . . . . . . . . . . . . . . . . . . .309
DNS Cache Snooping . . . . . . . . . . . . . . . . . . . . . .310
MITM Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . .311
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xxi
xxii Contents
Buffer Overflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . .313
Stack Overflows . . . . . . . . . . . . . . . . . . . . . . . . . . .314
Heap Overflows . . . . . . . . . . . . . . . . . . . . . . . . . . .320
Integer Overflows . . . . . . . . . . . . . . . . . . . . . . . . .320
Exploiting Buffer Overflows . . . . . . . . . . . . . . . . . .321
Reverse Engineering Code . . . . . . . . . . . . . . . . . . . . .324
Executable File Formats . . . . . . . . . . . . . . . . . . . . .325
Black-Box Analysis . . . . . . . . . . . . . . . . . . . . . . . . .327
White-Box Analysis . . . . . . . . . . . . . . . . . . . . . . . .329
Application Attack Platforms . . . . . . . . . . . . . . . . . . . .332
Metasploit Exploitation Framework . . . . . . . . . . . . .333
Other Application Attack Tools . . . . . . . . . . . . . . . .336
Defending the Application Layer . . . . . . . . . . . . . . . . . . . .336
SSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .336
SSH Protocol Architecture . . . . . . . . . . . . . . . . . . .336

Common Applications of SSH . . . . . . . . . . . . . . . .338
Pretty Good Privacy . . . . . . . . . . . . . . . . . . . . . . . . . .339
How PGP Works . . . . . . . . . . . . . . . . . . . . . . . . . .339
Key Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . .340
Securing Software . . . . . . . . . . . . . . . . . . . . . . . . . . . .340
Building Secure Software . . . . . . . . . . . . . . . . . . . .340
Security Testing Software . . . . . . . . . . . . . . . . . . . .341
Hardening Systems . . . . . . . . . . . . . . . . . . . . . . . . . . .343
Vulnerability Scanners . . . . . . . . . . . . . . . . . . . . . . . . .346
Nessus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .346
Application-Layer Security Project:
Using Nessus to Secure the Stack . . . . . . . . . . . . . . . . . . .347
Analyzing the Results . . . . . . . . . . . . . . . . . . . . . . . . .348
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .350
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .350
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .352
Chapter 9 Layer 8: The People Layer . . . . . . . . . . . . . . 353
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .354
Attacking the People Layer . . . . . . . . . . . . . . . . . . . . . . . .354
Social Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . .355
In Person . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .355
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xxii
Contents xxiii
Phone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .365
Fax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .366
Internet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .367
Phreaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .367
Phreak Boxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . .367
Wiretapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369
Stealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369

Cell Phones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369
World Wide Web, E-mail, and Instant Messaging . . . . .371
Trojan Horses and Backdoors . . . . . . . . . . . . . . . . .372
Disguising Programs . . . . . . . . . . . . . . . . . . . . . . . .372
Phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .372
Domain Name Spoofing . . . . . . . . . . . . . . . . . . . . .373
Secure Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . .374
Defending the People Layer . . . . . . . . . . . . . . . . . . . . . . . .375
Policies, Procedures, and Guidelines . . . . . . . . . . . . . . .375
Person-to-Person Authentication . . . . . . . . . . . . . . . . .377
Data Classification and Handling . . . . . . . . . . . . . . . . .377
Education,Training, and Awareness Programs . . . . . . . .378
Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .379
Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .381
Security Awareness Programs . . . . . . . . . . . . . . . . . .381
Evaluating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .382
Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .382
Monitoring and Enforcement . . . . . . . . . . . . . . . . . . .383
Periodic Update of Assessment and Controls . . . . . . . . .383
Regulatory Requirements . . . . . . . . . . . . . . . . . . . . . .383
Privacy Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383
Corporate Governance Laws . . . . . . . . . . . . . . . . . .386
Making the Case for Stronger Security . . . . . . . . . . . . . . .390
Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . .390
Asset Identification and Valuation . . . . . . . . . . . . . .390
Threat Assessment . . . . . . . . . . . . . . . . . . . . . . . . . .392
Impact Definition and Quantification . . . . . . . . . . .394
Control Design and Evaluation . . . . . . . . . . . . . . . .395
Residual Risk Management . . . . . . . . . . . . . . . . . .395
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xxiii

xxiv Contents
People Layer Security Project . . . . . . . . . . . . . . . . . . . . . .395
Orangebox Phreaking . . . . . . . . . . . . . . . . . . . . . . . . .396
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .398
Solutions Fast Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . .398
Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . .399
Appendix A Risk Mitigation: Securing the Stack. . . . . 401
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .402
Physical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .402
Data Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .403
Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .404
Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .405
Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .405
Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .406
Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .406
People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .420
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .422
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
408_Hack_the_Stack_TOC.qxd 9/22/06 7:50 PM Page xxiv
The first thing many people think of when they hear the word hack is some
type of malicious activity. I have always thought of the term in a somewhat
broader sense.Although some hacks are malicious, many others are not.
Nonmalicious hacks are about exploring the details of programmable systems
and learning how they really work.They are explored by those who want to
understand every minute detail of a system and how to stretch the capabilities
of these systems beyond what they were originally designed to do.The nonma-
licious hacker is different from the average user or even the script kiddie who
prefers to learn only the minimum necessary knowledge. Hack the Stack was
written for those who seek to better understand and to gain a deeper knowl-
edge of how TCP/IP systems really work. Such knowledge enables security

professionals to make systems and networks more secure and to meet the chal-
lenges that they face each day.
In Chapter 1, we provide you with information on how to extend OSI to
network security. In subsequent chapters, we unpeel the OSI onion layer by
layer, including a chapter on Layer 8 (the people layer).We conclude the book
with an appendix on risk mitigation.
Let’s talk about the writing of this book. Dedicated professionals like
George Mays, Stephen Watkins, Chris Ries, Ron Bandes, and Brandon Franklin
helped make this book possible. It takes a significant amount of time to com-
plete this type of task, and I am thankful to them for taking time out of their
daily work in the trenches to contribute to such an effort. After going through
this process more than once, my friends and family often ask how I have time
to work, travel, and then reserve time needed to write.Well, it takes time
xxv
Foreword
408_Hack_the_Stack_For.qxd 9/22/06 7:33 PM Page xxv