Unix power tools

•
Table of
Contents
• Index
• Reviews
•
Reader
Reviews
• Errata
UNIX Power Tools, 3rd Edition
By Shelley Powers, Jerry Peek, Tim O'Reilly,
Mike Loukides, et. al.

Publisher : O'Reilly
Pub Date : October 2002
ISBN : 0-596-00330-7
Pages : 1136
The latest edition of this best-selling favorite is loaded with vital
information on Linux, Darwin, and BSD. Unix Power Tools 3rd
Edition now offers more coverage of bash, zsh, and other new
shells, along with discussions about modern utilities and
applications. Several sections focus on security and Internet
access. There is a new chapter on access to Unix from
Windows, and expanded coverage of software installation and
packaging, as well as basic information on Perl and Python.
777
Copyright
How to Use This Book
Preface

A Book for Browsing
Like an Almanac
Like a News Magazine
Like a Hypertext Database
Programs on the Web
About Unix Versions
Cross-References
What's New in the Third Edition
Typefaces and Other Conventions
The Authors
The Fine Print
Request for Comments
Acknowledgments for the First Edition
Acknowledgments for the Second Edition
Acknowledgments for the Third Edition

Part I: Basic Unix Environment
Chapter 1. Introduction
Section 1.1. What's Special About Unix?
Section 1.2. Power Grows on You
Section 1.3. The Core of Unix
Section 1.4. Communication with Unix
Section 1.5. Programs Are Designed to Work Together
Section 1.6. There Are Many Shells
Section 1.7. Which Shell Am I Running?
Section 1.8. Anyone Can Program the Shell
Section 1.9. Internal and External Commands
Section 1.10. The Kernel and Daemons
Section 1.11. Filenames
Section 1.12. Filename Extensions

Section 1.13. Wildcards
Section 1.14. The Tree Structure of the Filesystem
Section 1.15. Your Home Directory
Section 1.16. Making Pathnames
Section 1.17. File Access Permissions
Section 1.18. The Superuser (Root)
Section 1.19. When Is a File Not a File?
Section 1.20. Scripting
Section 1.21. Unix Networking and Communications
Section 1.22. The X Window System

Chapter 2. Getting Help
Section 2.1. The man Command
Section 2.2. whatis: One-Line Command Summaries
Section 2.3. whereis: Finding Where a Command Is Located
Section 2.4. Searching Online Manual Pages
Section 2.5. How Unix Systems Remember Their Names
Section 2.6. Which Version Am I Using?
Section 2.7. What tty Am I On?
Section 2.8. Who's On?
Section 2.9. The info Command


Part II: Customizing Your Environment
Chapter 3. Setting Up Your Unix Shell
Section 3.1. What Happens When You Log In
Section 3.2. The Mac OS X Terminal Application
Section 3.3. Shell Setup Files — Which, Where, and Why
Section 3.4. Login Shells, Interactive Shells
Section 3.5. What Goes in Shell Setup Files?

Section 3.6. Tip for Changing Account Setup: Keep a Shell Ready
Section 3.7. Use Absolute Pathnames in Shell Setup Files
Section 3.8. Setup Files Aren't Read When You Want?
Section 3.9. Gotchas in set prompt Test
Section 3.10. Automatic Setups for Different Terminals
Section 3.11. Terminal Setup: Testing TERM
Section 3.12. Terminal Setup: Testing Remote Hostname and X Display
Section 3.13. Terminal Setup: Testing Port
Section 3.14. Terminal Setup: Testing Environment Variables
Section 3.15. Terminal Setup: Searching Terminal Table
Section 3.16. Terminal Setup: Testing Window Size
Section 3.17. Terminal Setup: Setting and Testing Window Name
Section 3.18. A .cshrc.$HOST File for Per Host Setup
Section 3.19. Making a "Login" Shell
Section 3.20. RC Files
Section 3.21. Make Your Own Manpages Without Learning troff
Section 3.22. Writing a Simple Manpage with the -man Macros

Chapter 4. Interacting with Your Environment
Section 4.1. Basics of Setting the Prompt
Section 4.2. Static Prompts
Section 4.3. Dynamic Prompts
Section 4.4. Simulating Dynamic Prompts
Section 4.5. C-Shell Prompt Causes Problems in vi, rsh, etc.
Section 4.6. Faster Prompt Setting with Built-ins
Section 4.7. Multiline Shell Prompts
Section 4.8. Session Info in Window Title or Status Line
Section 4.9. A "Menu Prompt" for Naive Users
Section 4.10. Highlighting and Color in Shell Prompts
Section 4.11. Right-Side Prompts

Section 4.12. Show Subshell Level with $SHLVL
Section 4.13. What Good Is a Blank Shell Prompt?
Section 4.14. dirs in Your Prompt: Better Than $cwd
Section 4.15. External Commands Send Signals to Set Variables
Section 4.16. Preprompt, Pre-execution, and Periodic Commands
Section 4.17. Running Commands When You Log Out
Section 4.18. Running Commands at Bourne/Korn Shell Logout
Section 4.19. Stop Accidental Bourne-Shell Logouts

Chapter 5. Getting the Most out of Terminals, xterm, and X Windows
Section 5.1. There's a Lot to Know About Terminals
Section 5.2. The Idea of a Terminal Database
Section 5.3. Setting the Terminal Type When You Log In
Section 5.4. Querying Your Terminal Type: qterm
Section 5.5. Querying Your xterm Size: resize
Section 5.6. Checklist: Terminal Hangs When I Log In
Section 5.7. Find Out Terminal Settings with stty
Section 5.8. Setting Your Erase, Kill, and Interrupt Characters
Section 5.9. Working with xterm and Friends
Section 5.10. Login xterms and rxvts
Section 5.11. Working with Scrollbars
Section 5.12. How Many Lines to Save?
Section 5.13. Simple Copy and Paste in xterm
Section 5.14. Defining What Makes Up a Word for Selection Purposes
Section 5.15. Setting the Titlebar and Icon Text
Section 5.16. The Simple Way to Pick a Font
Section 5.17. The xterm Menus
Section 5.18. Changing Fonts Dynamically
Section 5.19. Working with xclipboard
Section 5.20. Problems with Large Selections

Section 5.21. Tips for Copy and Paste Between Windows
Section 5.22. Running a Single Command with xterm -e
Section 5.23. Don't Quote Arguments to xterm -e

Chapter 6. Your X Environment
Section 6.1. Defining Keys and Button Presses with xmodmap
Section 6.2. Using xev to Learn Keysym Mappings
Section 6.3. X Resource Syntax
Section 6.4. X Event Translations
Section 6.5. Setting X Resources: Overview
Section 6.6. Setting Resources with the -xrm Option
Section 6.7. How -name Affects Resources
Section 6.8. Setting Resources with xrdb
Section 6.9. Listing the Current Resources for a Client: appres
Section 6.10. Starting Remote X Clients


Part III: Working with Files and Directories
Chapter 7. Directory Organization
Section 7.1. What? Me, Organized?
Section 7.2. Many Homes
Section 7.3. Access to Directories
Section 7.4. A bin Directory for Your Programs and Scripts
Section 7.5. Private (Personal) Directories
Section 7.6. Naming Files
Section 7.7. Make More Directories!
Section 7.8. Making Directories Made Easier

Chapter 8. Directories and Files
Section 8.1. Everything but the find Command

Section 8.2. The Three Unix File Times
Section 8.3. Finding Oldest or Newest Files with ls -t and ls -u
Section 8.4. List All Subdirectories with ls -R
Section 8.5. The ls -d Option
Section 8.6. Color ls
Section 8.7. Some GNU ls Features
Section 8.8. A csh Alias to List Recently Changed Files
Section 8.9. Showing Hidden Files with ls -A and -a
Section 8.10. Useful ls Aliases
Section 8.11. Can't Access a File? Look for Spaces in the Name
Section 8.12. Showing Nonprintable Characters in Filenames
Section 8.13. Counting Files by Types
Section 8.14. Listing Files by Age and Size
Section 8.15. newer: Print the Name of the Newest File
Section 8.16. oldlinks: Find Unconnected Symbolic Links
Section 8.17. Picking a Unique Filename Automatically

Chapter 9. Finding Files with find
Section 9.1. How to Use find
Section 9.2. Delving Through a Deep Directory Tree
Section 9.3. Don't Forget -print
Section 9.4. Looking for Files with Particular Names
Section 9.5. Searching for Old Files
Section 9.6. Be an Expert on find Search Operators
Section 9.7. The Times That find Finds
Section 9.8. Exact File-Time Comparisons
Section 9.9. Running Commands on What You Find
Section 9.10. Using -exec to Create Custom Tests
Section 9.11. Custom -exec Tests Applied
Section 9.12. Finding Many Things with One Command

Section 9.13. Searching for Files by Type
Section 9.14. Searching for Files by Size
Section 9.15. Searching for Files by Permission
Section 9.16. Searching by Owner and Group
Section 9.17. Duplicating a Directory Tree
Section 9.18. Using "Fast find" Databases
Section 9.19. Wildcards with "Fast find" Database
Section 9.20. Finding Files (Much) Faster with a find Database
Section 9.21. grepping a Directory Tree
Section 9.22. lookfor: Which File Has That Word?
Section 9.23. Using Shell Arrays to Browse Directories
Section 9.24. Finding the (Hard) Links to a File
Section 9.25. Finding Files with -prune
Section 9.26. Quick finds in the Current Directory
Section 9.27. Skipping Parts of a Tree in find
Section 9.28. Keeping find from Searching Networked Filesystem

Chapter 10. Linking, Renaming, and Copying Files
Section 10.1. What's So Complicated About Copying Files
Section 10.2. What's Really in a Directory?
Section 10.3. Files with Two or More Names
Section 10.4. More About Links
Section 10.5. Creating and Removing Links
Section 10.6. Stale Symbolic Links
Section 10.7. Linking Directories
Section 10.8. Showing the Actual Filenames for Symbolic Links
Section 10.9. Renaming, Copying, or Comparing a Set of Files
Section 10.10. Renaming a List of Files Interactively
Section 10.11. One More Way to Do It
Section 10.12. Copying Directory Trees with cp -r

Section 10.13. Copying Directory Trees with tar and Pipes

Chapter 11. Comparing Files
Section 11.1. Checking Differences with diff
Section 11.2. Comparing Three Different Versions with diff3
Section 11.3. Context diffs
Section 11.4. Side-by-Side diffs: sdiff
Section 11.5. Choosing Sides with sdiff
Section 11.6. Problems with diff and Tabstops
Section 11.7. cmp and diff
Section 11.8. Comparing Two Files with comm
Section 11.9. More Friendly comm Output
Section 11.10. make Isn't Just for Programmers!
Section 11.11. Even More Uses for make

Chapter 12. Showing What's in a File
Section 12.1. Cracking the Nut
Section 12.2. What Good Is a cat?
Section 12.3. "less" is More
Section 12.4. Show Nonprinting Characters with cat -v or od -c
Section 12.5. What's in That Whitespace?
Section 12.6. Finding File Types
Section 12.7. Squash Extra Blank Lines
Section 12.8. How to Look at the End of a File: tail
Section 12.9. Finer Control on tail
Section 12.10. How to Look at Files as They Grow
Section 12.11. GNU tail File Following
Section 12.12. Printing the Top of a File
Section 12.13. Numbering Lines

Chapter 13. Searching Through Files
Section 13.1. Different Versions of grep
Section 13.2. Searching for Text with grep
Section 13.3. Finding Text That Doesn't Match
Section 13.4. Extended Searching for Text with egrep
Section 13.5. grepping for a List of Patterns
Section 13.6. Approximate grep: agrep
Section 13.7. Search RCS Files with rcsgrep
Section 13.8. GNU Context greps
Section 13.9. A Multiline Context grep Using sed
Section 13.10. Compound Searches
Section 13.11. Narrowing a Search Quickly
Section 13.12. Faking Case-Insensitive Searches
Section 13.13. Finding a Character in a Column
Section 13.14. Fast Searches and Spelling Checks with "look"
Section 13.15. Finding Words Inside Binary Files
Section 13.16. A Highlighting grep

Chapter 14. Removing Files
Section 14.1. The Cycle of Creation and Destruction
Section 14.2. How Unix Keeps Track of Files: Inodes
Section 14.3. rm and Its Dangers
Section 14.4. Tricks for Making rm Safer
Section 14.5. Answer "Yes" or "No" Forever with yes
Section 14.6. Remove Some, Leave Some
Section 14.7. A Faster Way to Remove Files Interactively
Section 14.8. Safer File Deletion in Some Directories
Section 14.9. Safe Delete: Pros and Cons
Section 14.10. Deletion with Prejudice: rm -f
Section 14.11. Deleting Files with Odd Names

Section 14.12. Using Wildcards to Delete Files with Strange Names
Section 14.13. Handling a Filename Starting with a Dash (-)
Section 14.14. Using unlink to Remove a File with a Strange Name
Section 14.15. Removing a Strange File by its i-number
Section 14.16. Problems Deleting Directories
Section 14.17. Deleting Stale Files
Section 14.18. Removing Every File but One
Section 14.19. Using find to Clear Out Unneeded Files

Chapter 15. Optimizing Disk Space
Section 15.1. Disk Space Is Cheap
Section 15.2. Instead of Removing a File, Empty It
Section 15.3. Save Space with "Bit Bucket" Log Files and Mailboxes
Section 15.4. Save Space with a Link
Section 15.5. Limiting File Sizes
Section 15.6. Compressing Files to Save Space
Section 15.7. Save Space: tar and compress a Directory Tree
Section 15.8. How Much Disk Space?
Section 15.9. Compressing a Directory Tree: Fine-Tuning
Section 15.10. Save Space in Executable Files with strip
Section 15.11. Disk Quotas


Part IV: Basic Editing
Chapter 16. Spell Checking, Word Counting, and Textual Analysis
Section 16.1. The Unix spell Command
Section 16.2. Check Spelling Interactively with ispell
Section 16.3. How Do I Spell That Word?
Section 16.4. Inside spell
Section 16.5. Adding Words to ispell's Dictionary

Section 16.6. Counting Lines, Words, and Characters: wc
Section 16.7. Find a a Doubled Word
Section 16.8. Looking for Closure
Section 16.9. Just the Words, Please

Chapter 17. vi Tips and Tricks
Section 17.1. The vi Editor: Why So Much Material?
Section 17.2. What We Cover
Section 17.3. Editing Multiple Files with vi
Section 17.4. Edits Between Files
Section 17.5. Local Settings for vi
Section 17.6. Using Buffers to Move or Copy Text
Section 17.7. Get Back What You Deleted with Numbered Buffers
Section 17.8. Using Search Patterns and Global Commands
Section 17.9. Confirming Substitutions in vi
Section 17.10. Keep Your Original File, Write to a New File
Section 17.11. Saving Part of a File
Section 17.12. Appending to an Existing File
Section 17.13. Moving Blocks of Text by Patterns
Section 17.14. Useful Global Commands (with Pattern Matches)
Section 17.15. Counting Occurrences; Stopping Search Wraps
Section 17.16. Capitalizing Every Word on a Line
Section 17.17. Per-File Setups in Separate Files
Section 17.18. Filtering Text Through a Unix Command
Section 17.19. vi File Recovery Versus Networked Filesystems
Section 17.20. Be Careful with vi -r Recovered Buffers

Section 17.21. Shell Escapes: Running One UnixCommand While Using
Another
Section 17.22. vi Compound Searches

Section 17.23. vi Word Abbreviation

Section 17.24. Using vi Abbreviations as Commands (Cut and Paste
Between vi's)
Section 17.25. Fixing Typos with vi Abbreviations
Section 17.26. vi Line Commands Versus Character Commands
Section 17.27. Out of Temporary Space? Use Another Directory
Section 17.28. Neatening Lines
Section 17.29. Finding Your Place with Undo
Section 17.30. Setting Up vi with the .exrc File

Chapter 18. Creating Custom Commands in vi
Section 18.1. Why Type More Than You Have To?
Section 18.2. Save Time and Typing with the vi map Commands
Section 18.3. What You Lose When You Use map!
Section 18.4. vi @-Functions
Section 18.5. Keymaps for Pasting into a Window Running vi
Section 18.6. Protecting Keys from Interpretation by ex
Section 18.7. Maps for Repeated Edits
Section 18.8. More Examples of Mapping Keys in vi
Section 18.9. Repeating a vi Keymap
Section 18.10. Typing in Uppercase Without CAPS LOCK
Section 18.11. Text-Input Mode Cursor Motion with No Arrow Keys

Section 18.12. Don't Lose Important Functions with vi Maps: Use
noremap
Section 18.13. vi Macro for Splitting Long Lines
Section 18.14. File-Backup Macros

Chapter 19. GNU Emacs

Section 19.1. Emacs: The Other Editor
Section 19.2. Emacs Features: A Laundry List
Section 19.3. Customizations and How to Avoid Them
Section 19.4. Backup and Auto-Save Files
Section 19.5. Putting Emacs in Overwrite Mode
Section 19.6. Command Completion
Section 19.7. Mike's Favorite Timesavers
Section 19.8. Rational Searches
Section 19.9. Unset PWD Before Using Emacs
Section 19.10. Inserting Binary Characters into Files
Section 19.11. Using Word-Abbreviation Mode
Section 19.12. Directories for Emacs Hacks
Section 19.13. An Absurd Amusement

Chapter 20. Batch Editing
Section 20.1. Why Line Editors Aren't Dinosaurs
Section 20.2. Writing Editing Scripts
Section 20.3. Line Addressing
Section 20.4. Useful ex Commands
Section 20.5. Running Editing Scripts Within vi
Section 20.6. Change Many Files by Editing Just One
Section 20.7. ed/ex Batch Edits: A Typical Example
Section 20.8. Batch Editing Gotcha: Editors Fail on Big Files
Section 20.9. patch: Generalized Updating of Files That Differ
Section 20.10. Quick Reference: awk
Section 20.11. Versions of awk

Chapter 21. You Can't Quite Call This Editing
Section 21.1. And Why Not?
Section 21.2. Neatening Text with fmt

Section 21.3. Alternatives to fmt
Section 21.4. Clean Up Program Comment Blocks
Section 21.5. Remove Mail/News Headers with behead
Section 21.6. Low-Level File Butchery with dd
Section 21.7. offset: Indent Text
Section 21.8. Centering Lines in a File
Section 21.9. Splitting Files at Fixed Points: split
Section 21.10. Splitting Files by Context: csplit
Section 21.11. Hacking on Characters with tr
Section 21.12. Encoding "Binary" Files into ASCII
Section 21.13. Text Conversion with dd
Section 21.14. Cutting Columns or Fields
Section 21.15. Making Text in Columns with pr
Section 21.16. Make Columns Automatically with column
Section 21.17. Straightening Jagged Columns
Section 21.18. Pasting Things in Columns
Section 21.19. Joining Lines with join
Section 21.20. What Is (or Isn't) Unique?
Section 21.21. Rotating Text

Chapter 22. Sorting
Section 22.1. Putting Things in Order
Section 22.2. Sort Fields: How sort Sorts
Section 22.3. Changing the sort Field Delimiter
Section 22.4. Confusion with Whitespace Field Delimiters
Section 22.5. Alphabetic and Numeric Sorting
Section 22.6. Miscellaneous sort Hints
Section 22.7. lensort: Sort Lines by Length
Section 22.8. Sorting a List of People by Last Name

Part V: Processes and the Kernel
Chapter 23. Job Control
Section 23.1. Job Control in a Nutshell
Section 23.2. Job Control Basics
Section 23.3. Using jobs Effectively
Section 23.4. Some Gotchas with Job Control
Section 23.5. The "Current Job" Isn't Always What You Expect
Section 23.6. Job Control and autowrite: Real Timesavers!
Section 23.7. System Overloaded? Try Stopping Some Jobs
Section 23.8. Notification When Jobs Change State
Section 23.9. Stop Background Output with stty tostop
Section 23.10. nohup
Section 23.11. Disowning Processes
Section 23.12. Linux Virtual Consoles
Section 23.13. Stopping Remote Login Sessions

Chapter 24. Starting, Stopping, and Killing Processes
Section 24.1. What's in This Chapter
Section 24.2. fork and exec
Section 24.3. Managing Processes: Overall Concepts
Section 24.4. Subshells
Section 24.5. The ps Command
Section 24.6. The Controlling Terminal
Section 24.7. Tracking Down Processes
Section 24.8. Why ps Prints Some Commands in Parentheses
Section 24.9. The /proc Filesystem
Section 24.10. What Are Signals?
Section 24.11. Killing Foreground Jobs
Section 24.12. Destroying Processes with kill

Section 24.13. Printer Queue Watcher: A Restartable Daemon Shell
Script
Section 24.14. Killing All Your Processes
Section 24.15. Killing Processes by Name?
Section 24.16. Kill Processes Interactively
Section 24.17. Processes Out of Control? Just STOP Them
Section 24.18. Cleaning Up an Unkillable Process
Section 24.19. Why You Can't Kill a Zombie
Section 24.20. The Process Chain to Your Window
Section 24.21. Terminal Windows Without Shells
Section 24.22. Close a Window by Killing Its Process(es)

Chapter 25. Delayed Execution
Section 25.1. Building Software Robots the Easy Way
Section 25.2. Periodic Program Execution: The cron Facility
Section 25.3. Adding crontab Entries
Section 25.4. Including Standard Input Within a cron Entry
Section 25.5. The at Command
Section 25.6. Making Your at Jobs Quiet
Section 25.7. Checking and Removing Jobs
Section 25.8. Avoiding Other at and cron Jobs
Section 25.9. Waiting a Little While: sleep

Chapter 26. System Performance and Profiling
Section 26.1. Timing Is Everything
Section 26.2. Timing Programs

Section 26.3. What Commands Are Running and How Long Do They
Take?

Section 26.4. Checking System Load: uptime

Section 26.5. Know When to Be "nice" to Other Users and When Not
To
Section 26.6. A nice Gotcha
Section 26.7. Changing a Running Job's Niceness


Part VI: Scripting
Chapter 27. Shell Interpretation
Section 27.1. What the Shell Does
Section 27.2. How the Shell Executes Other Commands
Section 27.3. What's a Shell, Anyway?
Section 27.4. Command Evaluation and Accidentally Overwriting Files
Section 27.5. Output Command-Line Arguments One by One
Section 27.6. Controlling Shell Command Searches
Section 27.7. Wildcards Inside Aliases
Section 27.8. eval: When You Need Another Chance
Section 27.9. Which One Will bash Use?
Section 27.10. Which One Will the C Shell Use?
Section 27.11. Is It "2>&1 file" or "> file 2>&1"? Why?
Section 27.12. Bourne Shell Quoting
Section 27.13. Differences Between Bourne and C Shell Quoting
Section 27.14. Quoting Special Characters in Filenames
Section 27.15. Verbose and Echo Settings Show Quoting
Section 27.16. Here Documents
Section 27.17. "Special" Characters and Operators
Section 27.18. How Many Backslashes?

Chapter 28. Saving Time on the Command Line

Section 28.1. What's Special About the Unix Command Line
Section 28.2. Reprinting Your Command Line with CTRL-r
Section 28.3. Use Wildcards to Create Files?
Section 28.4. Build Strings with { }
Section 28.5. String Editing (Colon) Operators
Section 28.6. Automatic Completion
Section 28.7. Don't Match Useless Files in Filename Completion
Section 28.8. Repeating Commands
Section 28.9. Repeating and Varying Commands
Section 28.10. Repeating a Command with Copy-and-Paste
Section 28.11. Repeating a Time-Varying Command
Section 28.12. Multiline Commands, Secondary Prompts
Section 28.13. Here Document Example #1: Unformatted Form Letters
Section 28.14. Command Substitution
Section 28.15. Handling Lots of Text with Temporary Files
Section 28.16. Separating Commands with Semicolons
Section 28.17. Dealing with Too Many Arguments
Section 28.18. Expect

Chapter 29. Custom Commands
Section 29.1. Creating Custom Commands
Section 29.2. Introduction to Shell Aliases
Section 29.3. C-Shell Aliases with Command-Line Arguments
Section 29.4. Setting and Unsetting Bourne-Type Aliases
Section 29.5. Korn-Shell Aliases
Section 29.6. zsh Aliases
Section 29.7. Sourceable Scripts
Section 29.8. Avoiding C-Shell Alias Loops
Section 29.9. How to Put if-then-else in a C-Shell Alias
Section 29.10. Fix Quoting in csh Aliases with makealias and quote

Section 29.11. Shell Function Basics
Section 29.12. Shell Function Specifics
Section 29.13. Propagating Shell Functions
Section 29.14. Simulated Bourne Shell Functions and Aliases

Chapter 30. The Use of History
Section 30.1. The Lessons of History
Section 30.2. History in a Nutshell
Section 30.3. My Favorite Is !$
Section 30.4. My Favorite Is !:n*
Section 30.5. My Favorite Is ^^
Section 30.6. Using !$ for Safety with Wildcards
Section 30.7. History by Number
Section 30.8. History Substitutions
Section 30.9. Repeating a Cycle of Commands
Section 30.10. Running a Series of Commands on a File
Section 30.11. Check Your History First with :p
Section 30.12. Picking Up Where You Left Off
Section 30.13. Pass History to Another Shell
Section 30.14. Shell Command-Line Editing
Section 30.15. Changing History Characters with histchars
Section 30.16. Instead of Changing History Characters

Chapter 31. Moving Around in a Hurry
Section 31.1. Getting Around the Filesystem
Section 31.2. Using Relative and Absolute Pathnames
Section 31.3. What Good Is a Current Directory?
Section 31.4. How Does Unix Find Your Current Directory?
Section 31.5. Saving Time When You Change Directories: cdpath
Section 31.6. Loop Control: break and continue

Section 31.7. The Shells' pushd and popd Commands
Section 31.8. Nice Aliases for pushd
Section 31.9. Quick cds with Aliases
Section 31.10. cd by Directory Initials
Section 31.11. Finding (Anyone's) Home Directory, Quickly
Section 31.12. Marking Your Place with a Shell Variable
Section 31.13. Automatic Setup When You Enter/Exit a Directory

Chapter 32. Regular Expressions (Pattern Matching)
Section 32.1. That's an Expression
Section 32.2. Don't Confuse Regular Expressions with Wildcards
Section 32.3. Understanding Expressions
Section 32.4. Using Metacharacters in Regular Expressions
Section 32.5. Regular Expressions: The Anchor Characters ^ and $

Section 32.6. Regular Expressions: Matching a Character with a
Character Set
Section 32.7. Regular Expressions: Match Any Character with . (Dot)

Section 32.8. Regular Expressions: Specifying a Range of Characters
with [ ]
Section 32.9. Regular Expressions: Exceptions in a Character Set
Section 32.10. Regular Expressions: Repeating Character Sets with *

Section 32.11. Regular Expressions: Matching a Specific Number of Sets
with \ { and \ }
Section 32.12. Regular Expressions: Matching Words with \ < and \ >

Section 32.13. Regular Expressions: Remembering Patterns with \ (, \ ),
and \1

Section 32.14. Regular Expressions: Potential Problems
Section 32.15. Extended Regular Expressions
Section 32.16. Getting Regular Expressions Right
Section 32.17. Just What Does a Regular Expression Match?
Section 32.18. Limiting the Extent of a Match
Section 32.19. I Never Meta Character I Didn't Like
Section 32.20. Valid Metacharacters for Different Unix Programs
Section 32.21. Pattern Matching Quick Reference with Examples

Chapter 33. Wildcards
Section 33.1. File-Naming Wildcards
Section 33.2. Filename Wildcards in a Nutshell
Section 33.3. Who Handles Wildcards?
Section 33.4. What if a Wildcard Doesn't Match?
Section 33.5. Maybe You Shouldn't Use Wildcards in Pathnames
Section 33.6. Getting a List of Matching Files with grep -l
Section 33.7. Getting a List of Nonmatching Files
Section 33.8. nom: List Files That Don't Match a Wildcard

Chapter 34. The sed Stream Editor
Section 34.1. sed Sermon^H^H^H^H^H^HSummary
Section 34.2. Two Things You Must Know About sed
Section 34.3. Invoking sed
Section 34.4. Testing and Using a sed Script: checksed, runsed
Section 34.5. sed Addressing Basics
Section 34.6. Order of Commands in a Script
Section 34.7. One Thing at a Time
Section 34.8. Delimiting a Regular Expression
Section 34.9. Newlines in a sed Replacement
Section 34.10. Referencing the Search String in a Replacement

Section 34.11. Referencing Portions of a Search String
Section 34.12. Search and Replacement: One Match Among Many
Section 34.13. Transformations on Text
Section 34.14. Hold Space: The Set-Aside Buffer
Section 34.15. Transforming Part of a Line
Section 34.16. Making Edits Across Line Boundaries
Section 34.17. The Deliberate Scrivener
Section 34.18. Searching for Patterns Split Across Lines
Section 34.19. Multiline Delete
Section 34.20. Making Edits Everywhere Except
Section 34.21. The sed Test Command
Section 34.22. Uses of the sed Quit Command
Section 34.23. Dangers of the sed Quit Command
Section 34.24. sed Newlines, Quoting, and Backslashes in a Shell Script

Chapter 35. Shell Programming for the Uninitiated
Section 35.1. Writing a Simple Shell Program
Section 35.2. Everyone Should Learn Some Shell Programming
Section 35.3. What Environment Variables Are Good For
Section 35.4. Parent-Child Relationships
Section 35.5. Predefined Environment Variables
Section 35.6. The PATH Environment Variable
Section 35.7. PATH and path
Section 35.8. The DISPLAY Environment Variable
Section 35.9. Shell Variables
Section 35.10. Test String Values with Bourne-Shell case
Section 35.11. Pattern Matching in case Statements
Section 35.12. Exit Status of Unix Processes
Section 35.13. Test Exit Status with the if Statement
Section 35.14. Testing Your Success

Section 35.15. Loops That Test Exit Status
Section 35.16. Set Exit Status of a Shell (Script)
Section 35.17. Trapping Exits Caused by Interrupts
Section 35.18. read: Reading from the Keyboard
Section 35.19. Shell Script "Wrappers" for awk, sed, etc.
Section 35.20. Handling Command-Line Arguments in Shell Scripts
Section 35.21. Handling Command-Line Arguments with a for Loop
Section 35.22. Handling Arguments with while and shift
Section 35.23. Loop Control: break and continue
Section 35.24. Standard Command-Line Parsing
Section 35.25. The Bourne Shell set Command
Section 35.26. test: Testing Files and Strings
Section 35.27. Picking a Name for a New Command

Section 35.28. Finding a Program Name and Giving Your Program Multiple
Names
Section 35.29. Reading Files with the . and source Commands
Section 35.30. Using Shell Functions in Shell Scripts

Chapter 36. Shell Programming for the Initiated
Section 36.1. Beyond the Basics
Section 36.2. The Story of : # #!
Section 36.3. Don't Need a Shell for Your Script? Don't Use One
Section 36.4. Making #! Search the PATH
Section 36.5. The exec Command
Section 36.6. The Unappreciated Bourne Shell ":" Operator
Section 36.7. Parameter Substitution

Section 36.8. Save Disk Space and Programming: Multiple Names for a
Program

Section 36.9. Finding the Last Command-Line Argument
Section 36.10. How to Unset All Command-Line Parameters
Section 36.11. Standard Input to a for Loop
Section 36.12. Making a for Loop with Multiple Variables
Section 36.13. Using basename and dirname
Section 36.14. A while Loop with Several Loop Control Commands
Section 36.15. Overview: Open Files and File Descriptors
Section 36.16. n>&m: Swap Standard Output and Standard Error
Section 36.17. A Shell Can Read a Script from Its Standard Input, but
Section 36.18. Shell Scripts On-the-Fly from Standard Input
Section 36.19. Quoted hereis Document Terminators: sh Versus csh
Section 36.20. Turn Off echo for "Secret" Answers
Section 36.21. Quick Reference: expr
Section 36.22. Testing Characters in a String with expr
Section 36.23. Grabbing Parts of a String
Section 36.24. Nested Command Substitution
Section 36.25. Testing Two Strings with One case Statement
Section 36.26. Outputting Text to an X Window
Section 36.27. Shell Lockfile

Chapter 37. Shell Script Debugging and Gotchas
Section 37.1. Tips for Debugging Shell Scripts
Section 37.2. Bourne Shell Debugger Shows a Shell Variable
Section 37.3. Stop Syntax Errors in Numeric Tests
Section 37.4. Stop Syntax Errors in String Tests
Section 37.5. Quoting and Command-Line Parameters
Section 37.6. How Unix Keeps Time
Section 37.7. Copy What You Do with script
Section 37.8. Cleaning script Files
Section 37.9. Making an Arbitrary-Size File for Testing

Part VII: Extending and Managing Your Environment
Chapter 38. Backing Up Files
Section 38.1. What Is This "Backup" Thing?
Section 38.2. tar in a Nutshell
Section 38.3. Make Your Own Backups
Section 38.4. More Ways to Back Up
Section 38.5. How to Make Backups to a Local Device
Section 38.6. Restoring Files from Tape with tar
Section 38.7. Using tar to a Remote Tape Drive
Section 38.8. Using GNU tar with a Remote Tape Drive
Section 38.9. On-Demand Incremental Backups of a Project
Section 38.10. Using Wildcards with tar
Section 38.11. Avoid Absolute Paths with tar
Section 38.12. Getting tar's Arguments in the Right Order
Section 38.13. The cpio Tape Archiver
Section 38.14. Industrial Strength Backups

Chapter 39. Creating and Reading Archives
Section 39.1. Packing Up and Moving
Section 39.2. Using tar to Create and Unpack Archives
Section 39.3. GNU tar Sampler
Section 39.4. Managing and Sharing Files with RCS and CVS
Section 39.5. RCS Basics
Section 39.6. List RCS Revision Numbers with rcsrevs
Section 39.7. CVS Basics
Section 39.8. More CVS

Chapter 40. Software Installation

Section 40.1. /usr/bin and Other Software Directories
Section 40.2. The Challenges of Software Installation on Unix
Section 40.3. Which make?
Section 40.4. Simplifying the make Process
Section 40.5. Using Debian's dselect
Section 40.6. Installing Software with Debian's Apt-Get
Section 40.7. Interruptable gets with wget

Section 40.8. The curl Application and One-Step GNU-Darwin Auto-
Installer for OS X
Section 40.9. Installation with FreeBSD Ports
Section 40.10. Installing with FreeBSD Packages
Section 40.11. Finding and Installing RPM Packaged Software

Chapter 41. Perl
Section 41.1. High-Octane Shell Scripting
Section 41.2. Checking your Perl Installation
Section 41.3. Compiling Perl from Scratch
Section 41.4. Perl Boot Camp, Part 1: Typical Script Anatomy
Section 41.5. Perl Boot Camp, Part 2: Variables and Data Types
Section 41.6. Perl Boot Camp, Part 3: Branching and Looping
Section 41.7. Perl Boot Camp, Part 4: Pattern Matching
Section 41.8. Perl Boot Camp, Part 5: Perl Knows Unix
Section 41.9. Perl Boot Camp, Part 6: Modules
Section 41.10. Perl Boot Camp, Part 7: perldoc
Section 41.11. CPAN
Section 41.12. Make Custom grep Commands (etc.) with Perl
Section 41.13. Perl and the Internet

Chapter 42. Python

Section 42.1. What Is Python?
Section 42.2. Installation and Distutils
Section 42.3. Python Basics
Section 42.4. Python and the Web
Section 42.5. urllib
Section 42.6. urllib2
Section 42.7. htmllib and HTMLParser
Section 42.8. cgi
Section 42.9. mod_python
Section 42.10. What About Perl?


Part VIII: Communication and Connectivity
Chapter 43. Redirecting Input and Output
Section 43.1. Using Standard Input and Output
Section 43.2. One Argument with a cat Isn't Enough
Section 43.3. Send (Only) Standard Error Down a Pipe
Section 43.4. Problems Piping to a Pager
Section 43.5. Redirection in C Shell: Capture Errors, Too?
Section 43.6. Safe I/O Redirection with noclobber
Section 43.7. The ( ) Subshell Operators
Section 43.8. Send Output Two or More Places
Section 43.9. How to tee Several Commands into One Place
Section 43.10. Redirecting Output to More Than One Place
Section 43.11. Named Pipes: FIFOs
Section 43.12. What Can You Do with an Empty File?

Chapter 44. Devices
Section 44.1. Quick Introduction to Hardware
Section 44.2. Reading Kernel Boot Output

Section 44.3. Basic Kernel Configuration
Section 44.4. Disk Partitioning
Section 44.5. Filesystem Types and /etc/fstab
Section 44.6. Mounting and Unmounting Removable Filesystems
Section 44.7. Loopback Mounts
Section 44.8. Network Devices — ifconfig
Section 44.9. Mounting Network Filesystems — NFS, SMBFS
Section 44.10. Win Is a Modem Not a Modem?
Section 44.11. Setting Up a Dialup PPP Session
Section 44.12. USB Configuration
Section 44.13. Dealing with Sound Cards and Other Annoying Hardware
Section 44.14. Decapitating Your Machine — Serial Consoles

Chapter 45. Printing
Section 45.1. Introduction to Printing
Section 45.2. Introduction to Printing on Unix
Section 45.3. Printer Control with lpc
Section 45.4. Using Different Printers
Section 45.5. Using Symbolic Links for Spooling
Section 45.6. Formatting Plain Text: pr
Section 45.7. Formatting Plain Text: enscript
Section 45.8. Printing Over a Network
Section 45.9. Printing Over Samba
Section 45.10. Introduction to Typesetting
Section 45.11. A Bit of Unix Typesetting History
Section 45.12. Typesetting Manpages: nroff

Section 45.13. Formatting Markup Languages — troff, L
A
T

E
X, HTML, and
So On
Section 45.14. Printing Languages — PostScript, PCL, DVI, PDF
Section 45.15. Converting Text Files into a Printing Language
Section 45.16. Converting Typeset Files into a Printing Language
Section 45.17. Converting Source Files Automagically Within the Spooler
Section 45.18. The Common Unix Printing System (CUPS)
Section 45.19. The Portable Bitmap Package

Chapter 46. Connectivity
Section 46.1. TCP/IP — IP Addresses and Ports
Section 46.2. /etc/services Is Your Friend
Section 46.3. Status and Troubleshooting
Section 46.4. Where, Oh Where Did That Packet Go?
Section 46.5. The Director of Operations: inetd
Section 46.6. Secure Shell (SSH)
Section 46.7. Configuring an Anonymous FTP Server
Section 46.8. Mail — SMTP, POP, and IMAP
Section 46.9. Domain Name Service (DNS)
Section 46.10. Dynamic Host Configuration Protocol (DHCP)
Section 46.11. Gateways and NAT
Section 46.12. Firewalls
Section 46.13. Gatewaying from a Personal LAN over a Modem

Chapter 47. Connecting to MS Windows
Section 47.1. Building Bridges
Section 47.2. Installing and Configuring Samba
Section 47.3. Securing Samba
Section 47.4. SWAT and GUI SMB Browsers

Section 47.5. Printing with Samba
Section 47.6. Connecting to SMB Shares from Unix
Section 47.7. Sharing Desktops with VNC
Section 47.8. Of Emulators and APIs
Section 47.9. Citrix: Making Windows Multiuser


Part IX: Security
Chapter 48. Security Basics
Section 48.1. Understanding Points of Vulnerability
Section 48.2. CERT Security Checklists
Section 48.3. Keeping Up with Security Alerts
Section 48.4. What We Mean by Buffer Overflow
Section 48.5. What We Mean by DoS
Section 48.6. Beware of Sluggish Performance
Section 48.7. Intruder Detection
Section 48.8. Importance of MOTD
Section 48.9. The Linux proc Filesystem
Section 48.10. Disabling inetd
Section 48.11. Disallow rlogin and rsh
Section 48.12. TCP Wrappers

Chapter 49. Root, Group, and User Management
Section 49.1. Unix User/Group Infrastructure
Section 49.2. When Does a User Become a User
Section 49.3. Forgetting the root Password
Section 49.4. Setting an Exact umask
Section 49.5. Group Permissions in a Directory with the setgid Bit
Section 49.6. Groups and Group Ownership
Section 49.7. Add Users to a Group to Deny Permissions

Section 49.8. Care and Feeding of SUID and SGID Scripts
Section 49.9. Substitute Identity with su
Section 49.10. Never Log In as root
Section 49.11. Providing Superpowers with sudo
Section 49.12. Enabling Root in Darwin
Section 49.13. Disable logins

Chapter 50. File Security, Ownership, and Sharing
Section 50.1. Introduction to File Ownership and Security
Section 50.2. Tutorial on File and Directory Permissions
Section 50.3. Who Will Own a New File?
Section 50.4. Protecting Files with the Sticky Bit
Section 50.5. Using chmod to Change File Permission
Section 50.6. The Handy chmod = Operator
Section 50.7. Protect Important Files: Make Them Unwritable
Section 50.8. cx, cw, c-w: Quick File Permission Changes
Section 50.9. A Loophole: Modifying Files Without Write Access
Section 50.10. A Directory That People Can Access but Can't List
Section 50.11. Juggling Permissions
Section 50.12. File Verification with md5sum
Section 50.13. Shell Scripts Must Be Readable and (Usually) Executable
Section 50.14. Why Can't You Change File Ownership?
Section 50.15. How to Change File Ownership Without chown

Chapter 51. SSH
Section 51.1. Enabling Remote Access on Mac OS X
Section 51.2. Protecting Access Through SSH
Section 51.3. Free SSH with OpenSSH
Section 51.4. SSH Problems and Solutions
Section 51.5. General and Authentication Problems

Section 51.6. Key and Agent Problems
Section 51.7. Server and Client Problems


Glossary

Copyright
Copyright © 2003, 1997, 1993 O'Reilly & Associates, Inc. All rights reserved.
Printed in the United States of America.
Published by O'Reilly & Associates, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O'Reilly & Associates books may be purchased for educational, business, or sales promotional use.
Online editions are also available for most titles (). For more information contact
our corporate/institutional sales department: 800-998-9938 or
Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of
O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish
their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly &
Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial
caps. The association between the image of a hand drill and the topic of Unix is a trademark of O'Reilly
& Associates, Inc.
While every precaution has been taken in the preparation of this book, the publisher and the author assume
no responsibility for errors or omissions, or for damages resulting from the use of the information
contained herein.
How to Use This Book
This section refers to conventions used in the print book and explains how they were modified for the
Safari version. The numbers in the following images correspond to the list below.
1. Summary Boxes. You'll see gray shaded summary boxes all through the book. (On Safari, the
Summary Boxes are bordered sidebars with the title "Summary Box.") They summarize a topic
and point you to articles with examples and further explanation.
Article/Section Number. The first two digits indicate in which chapter the article resides; the last two
digits indicate the number of the article within that chapter. The article number is used to refer to this

article in all cross-references throughout the book. (On Safari, Article numbers correspond to Section
numbers.)
Cross-Reference in a Sentence. To find out more about the topic displayed in gray type (On Safari,
this text is displayed in boldface.), see the article referenced by the number in parentheses immediately
following the term.
Cross-Reference in a Code Example. When a cross-reference occurs in an example, the cross-
referenced text and related article number appear in the left margin. (On Safari, these cross-references
appear above the code example.)
5. Globe If you don't want to type this script into a file yourself, or if we're talking about a C
program that isn't shown, you can download it from the book's web site. See the Preface for
full details on the content available for download. (Online version available at
/>Screw. Be careful with this feature, or you might get screwed.
7. Pushpin. A note to keep in mind, or a helpful tip.
Bomb. A bomb icon in the margin is a cross-reference to another article that explains the possible
trouble you might encounter using the tip or script in the current article. (You can think of the bomb as a
cross-referenced screw.) (On Safari, the Bomb appears above the paragraph it refers to.)
Author's Initials. The author's full name is listed in the Preface.
Preface
A Book for Browsing
Like an Almanac
Like a News Magazine
Like a Hypertext Database
Programs on the Web
About Unix Versions
Cross-References
What's New in the Third Edition
Typefaces and Other Conventions
The Authors
The Fine Print
Request for Comments

Acknowledgments for the First Edition
Acknowledgments for the Second Edition
Acknowledgments for the Third Edition
A Book for Browsing
Technical books can be boring. But this is not an ordinary technical book! This book is like an almanac, a
news magazine, and a hypertext database all rolled into one. Instead of trying to put the topics in perfect
order — and expecting you to start at the beginning, then read through to the end — we hope that you'll
browse. Start anywhere. Read what you want. (That's not quite true. First, you should read this Preface
and the pages before it titled How to Use This Book. They will help you get the most out of your time with
this book. Next, you may want to skim through the Unix fundamentals in Chapter 1. Then read what you
want.)
Like an Almanac
The book is full of practical information. The main purpose isn't to teach you concepts (though they're in
here). We've picked a lot of common problems, and we'll show you how to solve them.
Even though it's not designed to be read in strict order, the book is organized into chapters with related
subject matter. If you want to find a specific subject, the table of contents is still a good place to start. In
addition, several of the chapters contain shaded boxes. These are like small tables of contents on a
particular subject, which might be even more limited than the scope of the chapter itself. Use the Index
when you're trying to find a specific piece of information instead of a general group of articles about a
topic.
Like a News Magazine
This book has short articles. Most show a problem and a solution — in one page or less. The articles are
numbered within each chapter. Not all articles are "how-to" tips. Some articles have background
information and concepts.
Like a Hypertext Database
Each article doesn't define all the concepts and words used. Instead, it gives you "links" that let you get
more information if you need it. It's easy to get more information when you need it, but just skip the link if
you don't. Unix Power Tools uses two kinds of links: those in a sentence and those in the margin. For
examples, see the pages before this Preface titled How to Use This Book.
Programs on the Web

The book describes scripts and freely available programs that are available on the web site. An article
about a program or file that's on the web site will have a globe icon next to it, like this. To get one of
these programs, use our visit the web site:
/>About Unix Versions
There are lots of similarities between different versions of Unix. But it's almost impossible to write a
book that covers every detail of every version correctly. Where we know there might be big differences
or problems, we'll print a note in the text. Other places, we're forced to use "weasel words" like "Some
versions of XXX will do ," without telling you exactly which versions. When you see those weasel
words, what can you do?
If the command or feature won't destroy anything when it doesn't work, try it! For instance, don't
experiment with rm, the command that removes files. But cat, a command that shows files, probably
won't hurt anything if some feature doesn't work with your version.
Look at the online manual or check your vendor's latest printed manuals. However, even these can
be wrong. For instance, your system administrator may have installed a local version of a command
that works differently — but not updated the online documentation. Be careful with "generic"
manuals, the kind you buy at a bookstore; there are lots of versions of Unix, and the manual may not
match your version closely enough.
Ask your system administrator or another "guru" for help before you use a command that might be
dangerous.
Cross-References
If a cross-reference is to a single word — for example, a command name like this: tar — the cross
reference is probably to an article that introduces that command. Cross references to phrases — like this:
from a parent process to child process — are to an article that explains more about the concept or
problem printed in gray.
Cross references don't necessarily give a complete list of all articles about a topic. We've tried to pick
one or a few articles that give the best information. For a more complete list, use the Index.
What's New in the Third Edition
There have been some big changes in Unix since we wrote the first edition in the early 1990s, and there's
been a surprising number of changes since the second edition, released in the late 1990s. Well over half
of the articles have been revised, and we've expanded our coverage of the so-called small Unix flavors:

Linux, FreeBSD, Mac OS X's Darwin, and so on.
A major change to this edition was the addition of several new topics relevant to today's connected
world, including protecting your machine from attack and several articles related to Internet protocols.
We've also added chapters with coverage of two of the more popular languages used in Unix: Perl and
Python.
Typefaces and Other Conventions
Italic
Is used for the names of all Unix utilities, switches, directories, and filenames and to emphasize new
terms and concepts when they are first introduced. It's also used in programs and examples to explain
what's happening or what's been left out at the . . . marks.
Bold
Is used occasionally within text to make words easy to find — just like movie stars' names in the