OReilly UNIX power tools 3rd edition oct 2002 ISBN 0596003307

CONTENTS


Chapter36.ShellProgrammingfortheInitiated
36.1BeyondtheBasics
36.2TheStoryof:##!
36.3Don'tNeedaShellforYourScript?Don'tUseOne
36.5TheexecCommand
36.6TheUnappreciatedBourneShell":"Operator
36.7ParameterSubstitution
36.8SaveDiskSpaceandProgramming:MultipleNamesfora
Program
36.9FindingtheLastCommand-LineArgument
36.10HowtoUnsetAllCommand-LineParameters
36.11StandardInputtoaforLoop
36.12MakingaforLoopwithMultipleVariables
36.13Usingbasenameanddirname
36.14AwhileLoopwithSeveralLoopControlCommands
36.15Overview:OpenFilesandFileDescriptors
36.16n>&m:SwapStandardOutputandStandardError
36.17AShellCanReadaScriptfromItsStandardInput,but...
36.18ShellScriptsOn-the-FlyfromStandardInput
36.19QuotedhereisDocumentTerminators:shVersuscsh
36.20TurnOffechofor"Secret"Answers
36.21QuickReference:expr
36.22TestingCharactersinaStringwithexpr
36.23GrabbingPartsofaString
36.24NestedCommandSubstitution
36.25TestingTwoStringswithOnecaseStatement
36.26OutputtingTexttoanXWindow
36.27ShellLockfile

36.1BeyondtheBasics


Thischapterhasabunchoftricksandtechniquesforprogrammingwiththe
Bourneshell.Someofthemaredocumentedbuthardtofind;othersaren't
documentedatall.Hereisasummaryofthischapter'sarticles:
Thefirstgroupofarticlesisaboutmakingafiledirectlyexecutablewith#!
onthefirstline.OnmanyversionsofUnix,anexecutablefilecanstartwith
afirstlinelikethis:

#!/path/to/interpreter
Thekernelwillstarttheprogramnamedinthatlineandgiveitthefileto
read.ChrisTorek'sUsenetclassic,Section36.2,explainshow#!started.
Section36.3explainsthatyour"shellscripts"maynotneedashellatall.
Thenextbunchofarticlesareaboutprocessesandcommands.Theexec
command,Section36.5,replacestheshellwithanotherprocess;itcanalso
beusedtochangeinput/outputredirection(seebelow).The:(colon)
operatorevaluatesitsargumentsandreturnsazerostatus—Section36.6
explainswhyyoushouldcare.
Nextaretechniquesforhandlingvariablesandparameters.Parameter
substitution,explainedinSection36.7,isacompactwaytotest,set,and
givedefaultvaluesforvariables.Youcanusethe$0parameterandUnix
linkstogivethesamescriptmultiplenamesandmakeitdomultiplethings;
seeSection36.8.Section36.9showstheeasywaytogetthelastcommandlineargument.Section36.10hasaneasywaytoremoveallthecommandlinearguments.
Fourarticlescovershloops.Aforloopusuallyreadsalistofsingle
argumentsintoasingleshellvariable.Section36.11showshowtomakethe
forloopreadfromstandardinput.Section36.12hastechniquesformaking
aforloopsetmorethanonevariable.Thedirnameandbasename
commandscanbeusedtosplitpathnameswithaloop;seeSection36.13.A

whileloopcanhavemorethanonecommandlineatthestart;seeSection
36.14.
Nextisanassortmentofarticlesaboutinput/output.Section36.15


introducesopenfilesandfiledescriptors—there'smoretoknowabout
standardinput/output/errorthanyoumighthaverealized!Section36.16has
alookatfile-descriptorhandlingintheBourneshell,swappingstandard
outputandstandarderror.
Theshellcanreadcommandsdirectlyfromashellscriptfile.AsSection
36.17pointsout,ashellcanalsoreadcommandsfromitsstandardinput,
butthatcancausesomeproblems.Section36.18showsoneplacescripts
fromstdinareuseful:writingascriptthatcreatesanotherscriptasitgoes.
NextaretwoarticlesaboutmiscellaneousI/O.Onegotchawiththeheredocumentoperator(forredirectinginputfromascriptfile)isthatthe
terminatorsaredifferentintheBourneandCshells;Section36.19explains.
Section36.20showshowtoturnoffechoingwhileyourscriptreadsa
"secret"answersuchasapassword.
Twoarticles—Section36.22andSection36.23—showusesforthe
versatileexprexpression-handlingcommand.Section36.21isaquick
referencetoexpr.Section36.24coversmultiplecommandsubstitution
(Section28.14).
Section36.25showsatrickformakingonecasestatement(Section35.10)
testtwothingsatonce.Finally,Section36.27hasasimpletechniquefor
gettingexclusiveaccesstoafileorothersystemresource.
—JP

36.2TheStoryof:##!
Onceuponatime,therewastheBourneshell.Sincetherewasonly"the"shell,
therewasnotroubledecidinghowtorunascript:runitwiththeshell.Itworked,
andeveryonewashappy.

Alongcameprogressandwroteanothershell.Thepeoplethoughtthiswasgood,
fornowtheycouldchoosetheirownshell.Sosomechosetheone,andsomethe
other,andtheywroteshellscriptsandwerehappy.Butonedaysomeonewho
usedthe"other"shellranascriptbysomeonewhousedthe"otherother"shell,


andalas!itbombedspectacularly.ThepeoplewailedandcalledupontheirGuru
forhelp.
"Well,"saidtheGuru,"Iseetheproblem.Theoneshellandtheotherarenot
compatible.Weneedtomakesurethattheshellsknowwhichothershelltouse
toruneachscript.Andlo!theoneshellhasa`comment'called:,andtheothera
truecommentcalled#.Iherebydecreethathenceforth,theoneshellwillrun
scriptsthatstartwith:,andtheotherthosethatstartwith#."Anditwasso,and
thepeoplewerehappy.
Butprogresswasnotfinished.Thistimehenoticedthatonlyshellsranscripts
andthoughtthatifthekerneltoocouldrunscripts,thiswouldbegood,andthe
peoplewouldbehappy.Sohewrotemorecode,andnowthekernelcouldrun
scriptsbutonlyiftheybeganwiththemagicincantation#!,andiftheytoldthe
kernelwhichshellranthescript.Anditwasso,andthepeoplewereconfused.
Forthe#!lookedlikea"comment."Thoughthekernelcouldseethe#!and
runashell,itwouldnotdosounlesscertainmagicbitswereset.Andifthe
incantationweremispronounced,thattoocouldstopthekernel,which,afterall,
wasnotomniscient.Andsothepeoplewailed,butalas!theGurudidnot
respond.Andsoitwas,andstillitistoday.Anyway,youwillgetbestresults
froma4BSDmachinebyusing

#!/bin/sh
or:

#!/bin/csh

asthefirstlineofyourscript.#!/bin/csh-fisalsohelpfulonoccasion,
andit'susuallyfasterbecausecshwon'treadyour.cshrcfile(Section3.3).
—CT

36.3Don'tNeedaShellforYourScript?Don't
UseOne


IfyourUnixunderstandsfilesthatstartwith:

#!/interpreter/program
(andnearlyallofthemdobynow)youdon'thavetousethoselinestostarta
shell,suchas#!/bin/sh.Ifyourscriptisjuststartingaprogramlikeawk,
Unixcanstarttheprogramdirectlyandsaveexecutiontime.Thisisespecially
usefulonsmalloroverloadedcomputers,orwhenyourscripthastobecalled
overandover(suchasinaloop).
First,herearetwoscripts.Bothscriptsprintthesecondwordfromeachlineof
textfiles.Oneusesashell;theotherrunsawkdirectly:

%catwith_sh
#!/bin/sh
awk'
{print$2}
'$*
%catno_sh
#!/usr/bin/awk-f
{print$2}
%catafile
onetwothreefourfive
Let'srunbothcommandsandtime(Section26.2)them.(Thisisrunningona

veryslowmachine.Onfastersystems,thisdifferencemaybehardertomeasure
—thoughthedifferencecanstilladdupovertime.)

%timewith_shafile
two
0.1u0.2s0:0026%
%timeno_shafile
two
0.0u0.1s0:0013%


Oneofthethingsthat'sreallyimportanttounderstandhereisthatwhenthe
kernelrunstheprogramontheinterpreterline,itisgiventhescript'sfilenameas
anargument.Iftheintepreterprogramunderstandsafiledirectly,like/bin/sh
does,nothingspecialneedstobedone.Butaprogramlikeawkorsedrequires
the-foptiontoreaditsscriptfromafile.Thisleadstotheseeminglyoddsyntax
intheexampleabove,withacalltoawk-fwithnofollowingfilename.The
scriptitselfistheinputfile!
Oneimplicationofthisusageisthattheinterpreterprogramneedstounderstand
#asacomment,orthefirstinterpreter-selectionlineitselfwillbeactedupon
(andprobablyrejectedby)theinterpreter.(Fortunately,theshells,Perl,sed,and
awk,amongotherprograms,dorecognizethiscommentcharacter.)
[Onelastcomment:ifyouhaveGNUtimeorsomeotherversionthathasa
verbosemode,youcanseethatthemajordifferencebetweenthetwoinvocations
isintermsofthepagefaultseachrequires.OnarelativelyspeedyPentium
III/450runningRedHatLinux,theversionusingashellastheinterpreter
requiredmorethantwicethemajorpagefaultsandmorethanthreetimesas
manyminorpagefaultsastheversioncallingawkdirectly.Onasystem,no
matterhowfast,thatisusingalargeamountofvirtualmemory,thesedifferences
canbecrucial.Sooptforperformance,andskiptheshellwhenit'snotneeded.

—SJC]
—JPandSJC
AsSection36.3explains,youcanuse#!/path/nametorunascriptwith
theinterpreterlocatedat/path/nameinthefilesystem.Theproblemcomesifa
newversionoftheinterpreterisinstalledsomewhereelseorifyourunthescript
onanothersystemthathasadifferentlocation.It'susuallynotaproblemfor
Bourneshellprogrammers:/bin/shexistsoneveryUnix-typesystemI'veseen.
Butsomenewershells—andinterpreterslikePerl—maybelurkingalmost
anywhere(althoughthisisbecomingmoreandmorestandardizedasPerland
othertoolslikeitbecomepartofstandardLinuxdistributionsandthelike).Ifthe
interpreterisn'tfound,you'llprobablygetacrypticmessagelike
scriptname:Commandnotfound,wherescriptnameisthename
ofthescriptfile.


TheenvcommandwillsearchyourPATH(Section35.6)foraninterpreter,then
execute(exec(Section24.2),replaceitself)withtheinterpreter.Ifyouwantto
trythis,typeenvls;envwillfindandrunlsforyou.Thisisprettyuseless
whenyouhaveashellaroundtointerpretyourcommands—becausetheshell
candothesamethingwithoutgettingenvinvolved.Butwhenthekernel
interpretsanexecutablefilethatstartswith#!,there'snoshell(yet!).That's
whereyoucanuseenv.Forinstance,torunyourscriptwithzsh,youcouldstart
itsfilewith:

#!/usr/bin/envzsh
...zshscripthere...
Thekernelexecs/usr/bin/env,thenenvfindsandexecsthezshitfound.Nice
trick,eh?Whatdoyouthinktheproblemis?(Youhavetenseconds...tick,tick,
tick...)Thecatchis:iftheenvcommandisn'tin/usr/binonyoursystem,this
trickwon'twork.Soit'snotasportableasitmightbe,butit'sstillhandyand

probablystillbetterthantryingtospecifythepathnameofalesscommon
interpreterlikezsh.
Runninganinterpreterthiswaycanalsobeasecurityproblem.Someone'sPATH
mightbewrong;forinstance,itmightexecutesomerandomcommandnamed
zshintheuser'sbindirectory.AnintrudercouldchangethePATHtomakethe
scriptuseacompletelydifferentinterpreterwiththesamename.
Onemoreproblemworthmentioning:youcan'tspecifyanyoptionsforthe
interpreteronthefirstline.Someshelloptionscanbesetlater,asthescript
starts,withacommandlikeset,shopt,andsoon—checktheshell'smanual
page.
Finally,understandthatusingenvlikethisprettymucherasesanyperformance
gainsyoumayhaveachievedusingthetrickinthepreviousarticle.
—JPandSJC

36.5TheexecCommand
Theexeccommandexecutesacommandinplaceofthecurrentshell;thatis,it


terminatesthecurrentshellandstartsanewprocess(Section24.3)initsplace.
Historically,execwasoftenusedtoexecutethelastcommandofashellscript.
Thiswouldkilltheshellslightlyearlier;otherwise,theshellwouldwaituntilthe
lastcommandwasfinished.Thispracticesavedaprocessandsomememory.
(Aren'tyougladyou'reusingamodernsystem?Thissortofconservationusually
isn'tnecessaryanylongerunlessyoursystemlimitsthenumberofprocesses
eachusercanhave.)
execcanbeusedtoreplaceoneshellwithanothershell:

%execksh
$
withoutincurringtheadditionaloverheadofhavinganunusedshellwaitingfor

thenewshelltofinish.
execalsomanipulatesfiledescriptors(Section36.16)intheBourneshell.
Whenyouuseexectomanagefiledescriptors,itdoesnotreplacethecurrent
process.Forexample,thefollowingcommandmakesthestandardinputofall
commandscomefromthefileformfileinsteadofthedefaultplace(usually,your
terminal):

exec—MLandJP

36.6TheUnappreciatedBourneShell":"
Operator
SomepeoplethinkthattheBourneshell's:isacommentcharacter.Itisn't,
really.Itevaluatesitsargumentsandreturnsazeroexitstatus(Section35.12).
Hereareafewplacestouseit:
ReplacetheUnixtruecommandtomakeanendlesswhileloop(Section
35.15).Thisismoreefficientbecausetheshelldoesn'thavetostartanew


processeachtimearoundtheloop(asitdoeswhenyouusewhile
true):

while:
do
commands
done
(Ofcourse,oneofthecommandswillprobablybebreak,toendtheloop
eventually.Thispresumesthatitisactuallyasavingstohavethebreaktest
insidetheloopbodyratherthanatthetop,butitmaywellbeclearerunder
certaincircumstancestodoitthatway.)

Whenyouwanttousetheelseinanif(Section35.13)butleavethethen
empty,the:makesanice"do-nothing"placefiller:

ifsomething
then:
else
commands
fi
IfyourBourneshelldoesn'thaveatrue#commentcharacter(butnearlyall
ofthemdonowadays),youcanuse:to"fakeit."It'ssafesttousequotesso
theshellwon'ttrytointerpretcharacterslike>or|inyour"comment":

:'readanswerandbranchif<3or>6'
Finally,it'susefulwithparametersubstitution(Section35.7)like
${var?}or${var=default}.Forinstance,usingthislineinyour
scriptwillprintanerrorandexitifeithertheUSERorHOMEvariables
aren'tset:

:${USER?}${HOME?}


—JP

36.7ParameterSubstitution
TheBourneshellhasahandysetofoperatorsfortestingandsettingshell
variables.They'relistedinTable36-1.
Table36-1.Bourneshellparametersubstitutionoperators
Operator

Explanation

${var:-default} Ifvarisnotsetorisempty,usedefaultinstead.
${var:=default} Ifvarisnotsetorisempty,setittodefaultandusethatvalue.
${var:+instead} Ifvarissetandisnotempty,useinstead.Otherwise,use
nothing(nullstring).

Ifvarissetandisnotempty,useitsvalue.Otherwise,print
${var:?message} message,ifany,andexitfromtheshell.Ifmessageismissing,
printadefaultmessage(whichdependsonyourshell).

Ifyouomitthecolon(:)fromtheexpressionsinTable36-1,theshelldoesn't
checkforanemptyparameter.Inotherwords,thesubstitutionhappens
whenevertheparameterisset.(That'showsomeearlyBourneshellswork:they
don'tunderstandacoloninparametersubstitution.)
Toseehowparametersubstitutionworks,here'sanotherversionofthebkedit
script(Section35.13,Section35.16):

+#!/bin/sh
ifcp"$1""$1.bak"
then


${VISUAL:-/usr/ucb/vi}"$1"
exit#Usestatusfromeditor
else
echo"`basename$0`quitting:can'tmakebackup?"1
exit1
fi
IftheVISUAL(Section35.5)environmentvariableissetandisnotempty,its
value(suchas/usr/local/bin/emacs)isusedandthecommandlinebecomes

/usr/local/bin/emacs"$1".IfVISUALisn'tset,thecommand
linedefaultsto/usr/ucb/vi"$1".
Youcanuseparametersubstitutionoperatorsinanycommandline.You'llsee
themusedwiththecolon(:)operator(Section36.6),checkingorsettingdefault
values.There'sanexamplebelow.Thefirstsubstitution
(${nothing=default})leaves$nothingemptybecausethe
variablehasbeenset.Thesecondsubstitutionsets$nothingtodefault
becausethevariablehasbeensetbutisempty.Thethirdsubstitutionleaves
$somethingsettostuff:

+nothing=
something=stuff
:${nothing=default}
:${nothing:=default}
:${something:=default}
SeveralBourne-typeshellshavesimilarstringeditingoperators,suchas
${var##pattern}.They'reusefulinshellprograms,aswellasonthe
commandlineandinshellsetupfiles.Seeyourshell'smanualpageformore
details.
—JP

36.8SaveDiskSpaceandProgramming:


MultipleNamesforaProgram
Ifyou'rewriting:
severalprogramsthatdothesamekindsofthings,
programsthatusealotofthesamecode(asyou'rewritingthesecond,third,
etc.,programs,youcopyalotoflinesfromthefirstprogram),or
aprogramwithseveraloptionsthatmakebigchangesinthewayitworks,

youmightwanttowritejustoneprogramandmakelinks(Section10.4,Section
10.3)toitinstead.Theprogramcanfindthenameyoucalleditwithand,
throughcaseortestcommands,workindifferentways.Forinstance,the
BerkeleyUnixcommandsex,vi,view,edit,andothersarealllinkstothesame
executablefile.Thistakeslessdiskspaceandmakesmaintenanceeasier.It's
usuallysensibleonlywhenmostofthecodeisthesameineachprogram.Ifthe
programisfullofnametestsandlotsofseparatecode,thistechniquemaybe
moretroublethanit'sworth.
Dependingonhowthescriptprogramiscalled,thisnamecanbeasimple
relativepathnamelikeprogor./prog—itcanalsobeanabsolute
pathnamelike/usr/joe/bin/prog(Section31.2explainspathnames).
Thereareacoupleofwaystohandlethisinashellscript.Ifthere'sjustonemain
pieceofcodeinthescript,asinthelfscript,acasethattests$0mightbebest.
Theasterisk(*)wildcardatthestartofeachcase(seeSection35.11)handlesthe
differentpathnamesthatmightbeusedtocallthescript:

case"$0"in
*name1)
...dothiswhencalledasname1...
;;
*name2)
...dothiswhencalledasname2...
;;


...
*)...printerrorandexitif$0doesn'tmatch...
;;
esac
Youmightalsowanttousebasename(Section36.13)tostripoffanyleading

pathnameandstorethecleaned-up$0inavariablecalledmyname.Youcantest
$mynameanywhereinthescriptandalsouseitforerrormessages:

myname=`basename$0`
...
case"$myname"in
...
echo"$myname:aborting;errorinxxxxxx"1>&2
...
—JP

36.9FindingtheLastCommand-LineArgument
Doyouneedtopickupthelastparameter$1,$2...fromtheparameterlist
onthecommandline?Itlookslikeeval\$$#woulddoit:

evalSection27.8
$setfoobarbaz
$evalecho\$$#
baz
exceptforasmallproblemwithshargumentsyntax:

$setmnopqrstuvwx
$echo$11


m1
$11means${1}1,not${11}.Trying${11}directlygivesbad
substitution.(Morerecentshells,suchasbash,dosupportthe${11}
syntax,however,toarbitrarylengths.Ourcopyofbash,forexample,allowedat
least10240commandlineargumentstosetwithrecallofthelastvia

${10240}).Yourmileagemayvary.
TheonlyreliablewaytogetatthelastparameterintheBourneshellistouse
somethinglikethis:

foridolast="$i";done
Theforloopassignseachparametertotheshellvariablenamedlast;afterthe
loopends,$lastwillhavethelastparameter.Also,notethatyouwon'tneed
thistrickonallsh-likeshells.TheKornshell,zsh,andbashunderstand${11}.
—CT

36.10HowtoUnsetAllCommand-Line
Parameters
Theshift(Section35.22)command"shiftsaway"onecommand-lineparameter.
Youcanshiftthreetimesiftherearethreecommand-lineparameters.Many
shellsalsocantakeanargument,likeshift3,thattellshowmanytimestoshift;
onthoseshells,youcanshift$#(Section35.20)tounsetallparameters.
Theportablewaytounsetallcommand-lineparametersisprobablytoset
(Section35.25)asingledummyparameter,thenshiftitaway:

+setx
shift
Settingthesingleparameterwipesoutwhateverotherparameterswereset
before.


—JP

36.11StandardInputtoaforLoop
AnobviousplacetouseaBourneshellforloop(Section35.21)istostep
throughalistofarguments—fromthecommandlineoravariable.Butcombine

theloopwithbackquotes(Section28.14)andcat(Section12.2),andtheloop
willstepthroughthewordsonstandardinput.
Here'sanexample:

forxin`cat`
do
...handle$x
done
Becausethismethodsplitstheinputintoseparatewords,nomatterhowmany
wordsareoneachinputline,itcanbemoreconvenientthanawhileloop
runningthereadcommand.Whenyouusethisscriptinteractively,though,the
loopwon'tstartrunninguntilyou'vetypedalloftheinput;usingwhilereadwill
runtheloopaftereachlineofinput.
—JP

36.12MakingaforLoopwithMultipleVariables
ThenormalBourneshellforloop(Section35.21)letsyoutakealistofitems,
storetheitemsonebyoneinashellvariable,andloopthroughasetof
commandsonceforeachitem:

forfileinprog1prog2prog3
do
...process$file
done


Iwantedaforloopthatstoresseveraldifferentshellvariablesandmakesone
passthroughtheloopforeachsetofvariables(insteadofonepassforeachitem,
asaregularforloopdoes).Thisloopdoesthejob:

setSection35.25

forbunchin"elliefile16""donnafile23""stevefile3
do
#PUTFIRSTWORD(USER)IN$1,SECOND(FILE)IN$2..
set$bunch
mail$1<$2
done
Ifyouhaveanycommand-lineargumentsandstillneedthem,storethemin
anothervariablebeforeyouusethesetcommand.Oryoucanmaketheloopthis
way:

forbunchin"u=ellief=file16s='yourfiles'"\
"u=donnaf=file23s='amemo'""u=stevef=file34s=r
do
#SET$u(USER),$f(FILENAME),$s(SUBJECT):
eval$bunch
mail-s"$s"$u<$f
done
Thisscriptusestheshell'seval(Section27.8)commandtorescanthecontentsof
thebunchvariableandstoreitinseparatevariables.Noticethesinglequotes,as
ins='yourfiles';thisgroupsthewordsforeval.Theshellremovesthe
singlequotesbeforeitstoresthevalueintothesvariable.
—JP

36.13Usingbasenameanddirname
AlmosteveryUnixcommandcanuserelativeandabsolutepathnames


(Section31.2)tofindafileordirectory.Therearetimesyou'llneedpartofa

pathname—thehead(everythingbeforethelastslash)orthetail(thenameafter
thelastslash).Theutilitiesbasenameanddirname,availableonmostUnix
systems,handlethat.

36.13.1Introductiontobasenameanddirname
Thebasenamecommandstripsany"path"namecomponentsfromafilename,
leavingyouwitha"pure"filename.Forexample:

%basename/usr/bin/gigiplot
gigiplot
%basename/home/mikel/bin/bvurns.sh
bvurns.sh
basenamecanalsostripasuffixfromafilename.Forexample:

%basename/home/mikel/bin/bvurns.sh.sh
bvurns
Thedirnamecommandstripsthefilenameitself,givingyouthe"directory"part
ofthepathname:

%dirname/usr/bin/screenblank
/usr/bin
%dirnamelocal
.
Ifyougivedirnamea"pure"filename(i.e.,afilenamewithnopath,asinthe
secondexample),ittellsyouthatthedirectoryis.(thecurrentdirectory).
dirnameandbasenamehaveabuginsomeimplementations.They
don'trecognizethesecondargumentasafilenamesuffixtostrip.
Here'sagoodtest:

%basename0.foo.foo
Iftheresultis0,yourbasenameimplementationisgood.Ifthe
answeris0.foo,theimplementationisbad.Ifbasenamedoesn't
work,dirnamewon't,either.

36.13.2UsewithLoops
Here'sanexampleofbasenameanddirname.There'sadirectorytreewithsome
verylargefiles—over100,000characters.Youwanttofindthosefiles,run
split(Section21.9)onthem,andaddhuge.tothestartoftheoriginalfilename.
Bydefault,splitnamesthefilechunksxaa,xab,xac,andsoon;youwanttouse
theoriginalfilenameandadot(.)insteadofx:

||Section35.14,exitSection35.16

forpathin`find/home/you-typef-size+100000c-pri
do
cd`dirname$path`||exit
filename=`basename$path`
split$filename$filename.
mv-i$filenamehuge.$filename
done
Thefindcommandwilloutputpathnameslikethese:

/home/you/somefile
/home/you/subdir/anotherfile
(Theabsolutepathnamesareimportanthere.Thecdwouldfailonthesecond
passoftheloopifyouuserelativepathnames.)Intheloop,thecdcommand
usesdirnametogotothedirectorywherethefileis.Thefilenamevariable,with
theoutputofbasename,isusedseveralplaces—twiceonthesplitcommand
line.

Ifthepreviouscoderesultsintheerrorcommandlinetoolong,
replacethefirstlineswiththetwolinesbelow.Thismakesaredirected-input
loop:

find/home/you-typef-size+100000c-print|
whilereadpath
—JPandML

36.14AwhileLoopwithSeveralLoopControl
Commands
IusedtothinkthattheBourneshell'swhileloop(Section35.15)lookedlike
this,withasinglecommandcontrollingtheloop:

whilecommand
do
...whatever
done
Butcommandcanactuallybealistofcommands.Theexitstatusofthelast
commandcontrolstheloop.Thisishandyforpromptingusersandreading
answers.Whentheusertypesanemptyanswer,thereadcommandreturns
"false"andtheloopends:

whileecho-e"EntercommandorCTRL-dtoquit:\c"
readcommand
do
...process$command
done
Youmayneeda-eoptiontomakeechotreatescapedcharacterslike\ctheway

youwant.Inthiscase,thecharacterringstheterminalbell,howeveryour
terminalinterpretsthat(oftenwithaflashofthescreen,forinstance.)


Here'saloopthatrunswhoanddoesaquicksearchonitsoutput.Ifthegrep
returnsnonzerostatus(becauseitdoesn'tfind$whoin$tempfile),the
loopquits—otherwise,theloopdoeslotsofprocessing:

while
who>$tempfile
grep"$who"$tempfile>/dev/null
do
...process$tempfile...
done
—JPandSJC

36.15Overview:OpenFilesandFileDescriptors
Thisintroductionisgeneralandsimplified.Ifyou'reatechnicalpersonwho
needsacompleteandexactdescription,readabookonUnixprogramming.
Unixshellsletyouredirecttheinputandoutputofprogramswithoperatorssuch
as>and|.Howdoesthatwork?Howcanyouuseitbetter?Here'sanoverview.
WhentheUnixkernelstartsanyprocess(Section24.3)—forexample,grep,ls,
orashell—itsetsupseveralplacesforthatprocesstoreadfromandwriteto,
asshowninFigure36-1.
Figure36-1.OpenstandardI/Ofileswithnocommand-line
redirection


Theseplacesarecalledopenfiles.Thekernelgiveseachfileanumbercalleda
filedescriptor.Butpeopleusuallyusenamesfortheseplacesinsteadofthe

numbers:
Thestandardinputorstdin(FileDescriptor(F.D.)number0)istheplace
wheretheprocesscanreadtext.Thismightbetextfromotherprograms
(throughapipe,onthecommandline)orfromyourkeyboard.
Thestandardoutputorstdout(F.D.1)isaplacefortheprocesstowriteits
results.
Thestandarderrororstderr(F.D.2)iswheretheprocesscansenderror
messages.
Bydefault,asFigure36-1shows,thefilethat'sopenedforstdin,stdout,and
stderris/dev/tty—anameforyourterminal.Thismakeslifeeasierforusers—
andprogrammers,too.Theuserdoesn'thavetotellaprogramwheretoreador
writebecausethedefaultisyourterminal.Aprogrammerdoesn'thavetoopen
filestoreadorwritefrom(inmanycases);theprogramscanjustreadfrom
stdin,writetostdout,andsenderrorstostderr.
Itgetsbetter.Whentheshellstartsaprocess(whenyoutypeacommandata
prompt),youcantelltheshellwhatfileto"connectto"anyofthosefile
descriptors.Forexample,Figure36-2showswhathappenswhenyourungrep
andmaketheshellredirectgrep'sstandardoutputawayfromtheterminaltoa
filenamedgrepout.
Figure36-2.Standardoutputredirectedtoafile


Programscanreadandwritefilesbesidestheonesonstdin,stdout,andstderr.
Forinstance,inFigure36-2,grepopenedthefilesomefileitself—itdidn'tuse
anyofthestandardfiledescriptorsforsomefile.AUnixconventionisthatifyou
don'tnameanyfilesonthecommandline,aprogramwillreadfromitsstandard
input.Programsthatworkthatwayarecalledfilters.
Allshellscandobasicredirectionwithstdin,stdout,andstderr.Butasyou'llsee
inSection36.16,theBourneshellalsohandlesfiledescriptors3through9(and
bashandtheothernewershellscanhandlearbitrarynumbersoffiledescriptiors,

uptowhateverulimit-nhappenstobeset).That'susefulsometimes:
Maybeyouhaveafewdatafilesthatyouwanttokeepreadingfromor
writingto.Insteadofgivingtheirnames,youcanusethefiledescriptor
numbers.
Onceyouopenafile,thekernelrememberswhatplaceinthefileyoulast
readfromorwroteto.Eachtimeyouusethatfiledescriptornumberwhile
thefileisopen,you'llbeatthesameplaceinthefile.That'sespeciallynice
whenyouwanttoreadfromorwritetothesamefilewithmorethanone
program.Forexample,thelinecommandonsomeUnixsystemsreadsone
linefromafile—youcancalllineoverandover,wheneveryouwantto
readthenextlinefromafile.Oncethefilehasbeenopened,youcan
removeitslink(name)fromthedirectory;theprocesscanaccessthefile
throughitsdescriptorwithoutusingthename.
WhenUnixstartsanewsubprocess(Section24.3),theopenfile


descriptorsaregiventothatprocess.Asubprocesscanreadorwritefrom
filedescriptorsopenedbyitsparentprocess.Aredirected-I/Oloop,as
discussedinSection43.6,takesadvantageofthis.
—JP

36.16n>&m:SwapStandardOutputand
StandardError
Bydefault,acommand'sstandarderrorgoestoyourterminal.Thestandard
outputgoestotheterminalorisredirectedsomewhere(toafile,downapipe,
intobackquotes).
Sometimesyouwanttheopposite.Forinstance,youmayneedtosenda
command'sstandardoutputtothescreenandgrabtheerrormessages(standard
error)withbackquotes.Oryoumightwanttosendacommand'sstandardoutput
toafileandthestandarderrordownapipetoanerror-processingcommand.

Here'showtodothatintheBourneshell.(TheCshellcan'tdothis,although
tcshcan.)
Filedescriptors0,1,and2are,respectively,thestandardinput,standardoutput,
andstandarderror(Section36.15explains).Withoutredirection,they'reall
associatedwiththeterminalfile/dev/tty(Section36.15).It'seasytoredirectany
descriptortoanyfile—ifyouknowthefilename.Forinstance,toredirectfile
descriptor2toerrfile,type:

$command2>errfile
Youknowthatapipeandbackquotesalsoredirectthestandardoutput:

$command|...
$var=`command`
Butthere'snofilenameassociatedwiththepipeorbackquotes,soyoucan'tuse
the2>redirection.Youneedtorearrangethefiledescriptorswithoutknowing
thefile(orwhatever)thatthey'reassociatedwith.Here'show.Youmayfindit


usefultorunthisshortPerlscript,whichsimplyprints"stdout"tostandard
output,and"stderr"tostandarderror:

#!/usr/bin/perl
printSTDOUT"stdout\n";
printSTDERR"stderr\n";
Let'sstartslowly.Wewillcombinebothstandardoutputandstandarderror,
sendingthembothasoutput,tobeusedastheinputtoapipeorastheoutputof
backquotes.TheBourneshelloperatorn>&mrearrangesthefilesandfile
descriptors.Itsays,"Makefiledescriptornpointtothesamefileasfile
descriptorm."Let'susethatoperatoronthepreviousexample.We'llsend
standarderrortothesameplacestandardoutputisgoing:

$command2>&1|...
$var=`command2>&1`
Inboththoseexamples,2>&1means"sendstandarderror(filedescriptor2)to
thesameplacestandardoutput(filedescriptor1)isgoing."Simple,eh?
Youcanusemorethanonen>&moperator.Theshellreadsthemleft-to-right
beforeitexecutesthecommand.
"Oh!"youmightsay."Toswapstandardoutputandstandarderror—make
stderrgodownapipeandstdoutgotothescreen—Icoulddothis!"

$command2>&11>&2|...wrong...
Sorry,Charlie.Whentheshellsees2>&11>&2,theshellfirstdoes2>&1.
You'veseenthatbefore—itmakesfiledescriptor2(stderr)gothesameplace
asfiledescriptor1(stdout).Thentheshelldoes1>&2.Itmakesstdout(1)go
thesameplaceasstderr(2)...butstderrisalreadygoingthesameplaceas
stdout,downthepipe.
Thisisoneplacetheotherfiledescriptors,3through9(andhigherinbash),