Learning the bash shell - unix shell programming

1
Learning the bash Shell,
3rd Edition
Table of Contents
2
Preface
bash Versions
Summary of bash Features
Intended Audience
Code Examples
Chapter Summary
Conventions Used in This Handbook
We'd Like to Hear from You
Using Code Examples
Safari Enabled
Acknowledgments for the First Edition
Acknowledgments for the Second Edition
Acknowledgments for the Third Edition
1. bash Basics
3
1.1. What Is a Shell?
1.2. Scope of This Book
1.3. History of UNIX Shells
1.3.1. The Bourne Again Shell
1.3.2. Features of bash
1.4. Getting bash
1.5. Interactive Shell Use
1.5.1. Commands, Arguments, and Options
1.6. Files
1.6.1. Directories
1.6.2. Filenames, Wildcards, and Pathname

Expansion
1.6.3. Brace Expansion
1.7. Input and Output
1.7.1. Standard I/O
1.7.2. I/O Redirection
1.7.3. Pipelines
1.8. Background Jobs
1.8.1. Background I/O
1.8.2. Background Jobs and Priorities
1.9. Special Characters and Quoting
1.9.1. Quoting
1.9.2. Backslash-Escaping
1.9.3. Quoting Quotation Marks
1.9.4. Continuing Lines
1.9.5. Control Keys
4
1.10. Help
2. Command-Line Editing
2.1. Enabling Command-Line Editing
2.2. The History List
2.3. emacs Editing Mode
2.3.1. Basic Commands
2.3.2. Word Commands
2.3.3. Line Commands
2.3.4. Moving Around in the History List
2.3.5. Textual Completion
2.3.6. Miscellaneous Commands
2.4. vi Editing Mode
2.4.1. Simple Control Mode Commands
2.4.2. Entering and Changing Text

2.4.3. Deletion Commands
2.4.4. Moving Around in the History List
2.4.5. Character-Finding Commands
2.4.6. Textual Completion
2.4.7. Miscellaneous Commands
2.5. The fc Command
2.6. History Expansion
2.7. readline
2.7.1. The readline Startup File
2.7.2. Key Bindings Using bind
2.8. Keyboard Habits
3. Customizing Your Environment
5
3.1. The .bash_profile, .bash_logout, and .bashrc
Files
3.2. Aliases
3.3. Options
3.3.1. shopt
3.4. Shell Variables
3.4.1. Variables and Quoting
3.4.2. Built-In Variables
3.5. Customization and Subprocesses
3.5.1. Environment Variables
3.5.2. The Environment File
3.6. Customization Hints
4. Basic Shell Programming
6
4.1. Shell Scripts and Functions
4.1.1. Functions
4.2. Shell Variables

4.2.1. Positional Parameters
4.2.2. Local Variables in Functions
4.2.3. Quoting with $@ and $*
4.2.4. More on Variable Syntax
4.3. String Operators
4.3.1. Syntax of String Operators
4.3.2. Patterns and Pattern Matching
4.3.3. Length Operator
4.3.4. Extended Pattern Matching
4.4. Command Substitution
4.5. Advanced Examples: pushd and popd
5. Flow Control
5.1. if/else
5.1.1. Exit Status
5.1.2. Return
5.1.3. Combinations of Exit Statuses
5.1.4. Condition Tests
5.1.5. Integer Conditionals
5.2. for
5.3. case
5.4. select
5.5. while and until
6. Command-Line Options and Typed Variables
7
6.1. Command-Line Options
6.1.1. shift
6.1.2. Options with Arguments
6.1.3. getopts
6.2. Typed Variables
6.3. Integer Variables and Arithmetic

6.3.1. Arithmetic Conditionals
6.3.2. Arithmetic Variables and Assignment
6.3.3. Arithmetic for Loops
6.4. Arrays
7. Input/Output and Command-Line Processing
7.1. I/O Redirectors
7.1.1. Here-documents
7.1.2. File Descriptors
7.2. String I/O
7.2.1. echo
7.2.2. printf
7.2.3. read
7.3. Command-Line Processing
7.3.1. Quoting
7.3.2. command, builtin, and enable
7.3.3. eval
8. Process Handling
8
8.1. Process IDs and Job Numbers
8.2. Job Control
8.2.1. Foreground and Background
8.2.2. Suspending a Job
8.3. Signals
8.3.1. Control-Key Signals
8.3.2. kill
8.3.3. ps
8.4. trap
8.4.1. Traps and Functions
8.4.2. Process ID Variables and Temporary
Files

8.4.3. Ignoring Signals
8.4.4. disown
8.4.5. Resetting Traps
8.5. Coroutines
8.5.1. wait
8.5.2. Advantages and Disadvantages of
Coroutines
8.5.3. Parallelization
8.6. Subshells
8.6.1. Subshell Inheritance
8.6.2. Nested Subshells
8.7. Process Substitution
9. Debugging Shell Programs
9
9.1. Basic Debugging Aids
9.1.1. Set Options
9.1.2. Fake Signals
9.1.3. Debugging Variables
9.2. A bash Debugger
9.2.1. Structure of the Debugger
9.2.2. The Preamble
9.2.3. Debugger Functions
9.2.4. A Sample bashdb Session
9.2.5. Exercises
10. bash Administration
10.1. Installing bash as the Standard Shell
10.1.1. POSIX Mode
10.1.2. Command-Line Options
10.2. Environment Customization
10.2.1. umask

10.2.2. ulimit
10.2.3. Types of Global Customization
10.3. System Security Features
10.3.1. Restricted Shell
10.3.2. A System Break-In Scenario
10.3.3. Privileged Mode
11. Shell Scripting
10
11.1. What's That Do?
11.1.1. Comments
11.1.2. Variables and Constants
11.2. Starting Up
11.3. Potential Problems
11.4. Don't Use bash
12. bash for Your System
12.1. Obtaining bash
12.2. Unpacking the Archive
12.3. What's in the Archive
12.3.1. Documentation
12.3.2. Configuring and Building bash
12.3.3. Testing bash
12.3.4. Potential Problems
12.3.5. Installing bash as a Login Shell
12.3.6. Examples
12.4. Who Do I Turn to?
12.4.1. Asking Questions
12.4.2. Reporting Bugs
A. Related Shells
11
A.1. The Bourne Shell

A.2. The IEEE 1003.2 POSIX Shell Standard
A.3. The Korn Shell
A.4. pdksh
A.5. zsh
A.6. Shell Clones and Unix-like Platforms
A.6.1. Cygwin
A.6.2. DJGPP
A.6.3. MKS Toolkit
A.6.4. AT&T UWIN
B. Reference Lists
B.1. Invocation
B.2. Prompt String Customizations
B.3. Built-In Commands and Reserved Words
B.4. Built-In Shell Variables
B.5. Test Operators
B.6. set Options
B.7. shopt Options
B.8. I/O Redirection
B.9. emacs Mode Commands
B.10. vi Control Mode Commands
C. Loadable Built-Ins
D. Programmable Completion
12
Learning the bash Shell,
3rd Edition
Cameron Newham
Copyright © 2009 O'Reilly Media, Inc.
O'Reilly Media, Inc.
13
Preface

The first thing users of the UNIX or Linux operating
systems come face to face with is the shell. "Shell" is the
UNIX term for a user interface to the system—something
that lets you communicate with the computer via the
keyboard and the display. Shells are just separate
programs that encapsulate the system, and, as such, there
are many to choose from.
Systems are usually set up with a "standard" shell that
new users adopt without question. However, some of
these standard shells are rather old and lack many
features of the newer shells. This is a shame, because
shells have a large bearing on your working environment.
Since changing shells is as easy as changing hats, there is
no reason not to change to the latest and greatest in shell
technology.
Of the many shells to choose from, this book introduces
the Bourne Again shell (bash for short), a modern
general-purpose shell. Other useful modern shells are the
Korn shell (ksh) and the "Tenex C shell" (tcsh); both are
also the subjects of O'Reilly handbooks.
14
bash Versions
This book is relevant to all versions of bash, although
older versions lack some of the features of the most
recent version.
[1]
You can easily find out which version
you are using by typing echo $BASH_VERSION. The
earliest public version of bash was 1.0, and the most
recent is 3.0 (released in July 2004). If you have an older

version, you might like to upgrade to the latest one.
Chapter 12 shows you how to go about it.
[1]
Throughout this book we have clearly marked with
footnotes the features that are not present in the earlier
versions.
15
Summary of bash
Features
bash is a backward-compatible evolutionary successor to
the Bourne shell that includes most of the C shell's major
advantages as well as features from the Korn shell and a
few new features of its own. Features appropriated from
the C shell include:
• Directory manipulation, with the pushd, popd,
and dirs commands.
• Job control, including the fg and bg commands
and the ability to stop jobs with CTRL-Z.
• Brace expansion, for generating arbitrary strings.
• Tilde expansion, a shorthand way to refer to
directories.
• Aliases, which allow you to define shorthand
names for commands or command lines.
• Command history, which lets you recall
previously entered commands.
bash's major new features include:
16
• Command-line editing, allowing you to use vi- or
emacs-style editing commands on your command
lines.

• Key bindings that allow you to set up customized
editing key sequences.
• Integrated programming features: the
functionality of several external UNIX
commands, including test, expr, getopt, and echo,
has been integrated into the shell itself, enabling
common programming tasks to be done more
cleanly and efficiently.
• Control structures, especially the select construct,
which enables easy menu generation.
• New options and variables that give you more
ways to customize your environment.
• One dimensional arrays that allow easy
referencing and manipulation of lists of data.
• Dynamic loading of built-ins, plus the ability to
write your own and load them into the running
shell.
17
Intended Audience
This book is designed to address casual UNIX and Linux
users who are just above the "raw beginner" level. You
should be familiar with the process of logging in, entering
commands, and doing simple things with files. Although
Chapter 1 reviews concepts such as the tree-like file and
directory scheme, you may find that it moves too quickly
if you're a complete neophyte. In that case, we
recommend the O'Reilly handbook, Learning the UNIX
Operating System, by Jerry Peek, Grace Todino, and John
Strang.
If you're an experienced user, you may wish to skip

Chapter 1 altogether. But if your experience is with the C
shell, you may find that Chapter 1 reveals a few subtle
differences between the bash and C shells.
No matter what your level of experience is, you will
undoubtedly learn many things in this book that will
make you a more productive bash user—from major
features down to details at the "nook-and-cranny" level
that you may not have been aware of.
If you are interested in shell programming (writing shell
scripts and functions that automate everyday tasks or
serve as system utilities), you should also find this book
useful. However, we have deliberately avoided drawing a
strong distinction between interactive shell use (entering
18
commands during a login session) and shell
programming. We see shell programming as a natural,
inevitable outgrowth of increasing experience as a user.
Accordingly, each chapter depends on those previous to
it, and although the first three chapters are oriented
toward interactive use only, subsequent chapters describe
interactive, user-oriented features in addition to
programming concepts.
This book aims to show you that writing useful shell
programs doesn't require a computing degree. Even if you
are completely new to computing, there is no reason why
you shouldn't be able to harness the power of bash within
a short time.
Toward that end, we decided not to spend too much time
on features of exclusive interest to low-level systems
programmers. Concepts like file descriptors and special

file types might only confuse the casual user, and
anyway, we figure those of you who understand such
things are smart enough to extrapolate the necessary
information from our cursory discussions.
19
Code Examples
This book is full of examples of shell commands and
programs designed to be useful in your everyday life as a
user, not just to illustrate the feature being explained. In
Chapter 4 and onwards, we include various programming
problems, which we call tasks, that illustrate particular
shell programming concepts. Some tasks have solutions
that are refined in subsequent chapters. The later chapters
also include programming exercises, many of which build
on the tasks in the chapter.
Feel free to use any code you see in this book and to pass
it along to friends and colleagues. We especially
encourage you to modify and enhance it yourself.
If you want to try examples but you don't use bash as
your login shell, you must put the following line at the
top of each shell script:
#!/bin/bash
If bash isn't installed as the file /bin/bash, substitute its
pathname in the above.
20
Chapter Summary
If you want to investigate specific topics rather than read
the entire book through, here is a chapter-by-chapter
summary:
Chapter 1 introduces bash and tells you how to install it

as your login shell. Then it surveys the basics of
interactive shell use, including overviews of the UNIX
file and directory scheme, standard I/O, and background
jobs.
Chapter 2 discusses the shell's command history
mechanism (including the emacs- and vi-editing modes),
history substitution and the fc history command, and key
bindings with readline and bind.
Chapter 3 covers ways to customize your shell
environment without programming by using the startup
and environment files. Aliases, options, and shell
variables are the customization techniques discussed.
Chapter 4 is an introduction to shell programming. It
explains the basics of shell scripts and functions, and
discusses several important "nuts-and-bolts"
programming features: string manipulation operators,
brace expansion, command-line arguments (positional
parameters), and command substitution.
21
Chapter 5 continues the discussion of shell programming
by describing command exit status, conditional
expressions, and the shell's flow-control structures: if,
for, case, select, while, and until.
Chapter 6 goes into depth about positional parameters and
command-line option processing, then discusses special
types and properties of variables, integer arithmetic, and
arrays.
Chapter 7 gives a detailed description of bash I/O. This
chapter covers all of the shell's I/O redirectors, as well as
the line-at-a-time I/O commands read and echo. It also

discusses the shell's command-line processing mechanism
and the eval command.
Chapter 8 covers process-related issues in detail. It starts
with a discussion of job control, then gets into various
low-level information about processes, including process
IDs, signals, and traps. The chapter then moves to a
higher level of abstraction to discuss coroutines and
subshells.
Chapter 9 discusses various debugging techniques, like
trace and verbose modes, and the "fake" signal traps. It
then presents in detail a useful shell tool, written using
the shell itself: a bash debugger.
Chapter 10 gives information for system administrators,
including techniques for implementing system-wide shell
customization and features related to system security.
22
Chapter 11 discusses ways to make bash scripts more
maintainable.
Chapter 12 shows you how to go about getting bash and
how to install it on your system. It also outlines what to
do in the event of problems along the way.
Appendix A compares bash to several similar shells,
including the standard Bourne shell, the POSIX shell
standard, the Korn shell (ksh), the public-domain Korn
shell (pdksh), and the Z Shell (zsh).
Appendix B contains lists of shell invocation options,
built-in commands, built-in variables, conditional test
operators, options, I/O redirection, and emacs- and
vi-editing mode commands.
Appendix C gives information on writing and compiling

your own loadable built-ins.
Appendix D looks at the basics of programmable
completion.
23
Conventions Used in
This Handbook
We leave it as understood that when you enter a shell
command, you press RETURN at the end. RETURN is
labeled ENTER on some keyboards.
Characters called CTRL-X, where X is any letter, are
entered by holding down the CTRL (or CTL, or
CONTROL) key and pressing that letter. Although we
give the letter in uppercase, you can press the letter
without the SHIFT key.
Other special characters are LINEFEED (which is the
same as CTRL-J), BACKSPACE (same as CTRL-H),
ESC, TAB, and DEL (sometimes labeled DELETE or
RUBOUT).
This book uses the following font conventions:
Italic
Used for UNIX filenames, commands not built into
the shell (which are files anyway), and shell
functions. Italic is also used for dummy parameters
that should be replaced with an actual value, to
distinguish the vi and emacs programs from their
bash modes, and to highlight special terms the first
time they are defined.
24
Bold
Used for bash built-in commands, aliases, variables,

and options, as well as command lines when they are
within regular text. Bold is used for all elements
typed in by the user within regular text.
Constant Width
Used in examples to show the contents of files or the
output from commands.
Constant Bold
Used in examples to show interaction between the
user and the shell; any text the user types in is shown
in Constant Bold. For example:$ pwd/home/
cam/adventure/carrol $
Constant Italic
Used in displayed command lines for dummy
parameters that should be replaced with an actual
value.
Square Brackets
Used in Chapter 2 to show the position of the cursor
on the command line being edited. For
example:grep -l Alice < ~cam/book/
[a]iw
25