Linz an introduction to formal languages and automata jones bartlett (2012)

AnIntroductionto

FORMALLANGUAGES
andAUTOMATA
FifthEdition

PETERLINZ
UniversityofCaliforniaatDavis

JONES&BARTLETT

LEARNING


WorldHeadquarters
Jones&BartlettLearning
40TallPineDrive
Sudbury,MA01776
978-443-5000

www.jblearning.com

Jones&BartlettLearning
Canada
6339OrmindaleWay
Mississauga,OntarioL5V
1J2
Canada

Jones&BartlettLearning
International
BarbHouse,BarbMews
LondonW67PA
UnitedKingdom

Jones&BartlettLearningbooksandproductsareavailablethroughmostbookstoresandonlinebooksellers.Tocontact
Jones&BartlettLearningdirectly,call800-832-0034,fax978-443-8000,orvisitourwebsite,www.jblearning.com.
Substantial discounts on bulk quantities of Jones & Bartlett Learning publications are available to corporations,
professionalassociations,andotherqualifiedorganizations.Fordetailsandspecificdiscountinformation,contactthe
special sales department at Jones & Bartlett Learning via the above contact information or send an email to

Copyright©2012byJones&BartlettLearning,LLC
All rights reserved. No part of the material protected by this copyright may be reproduced or utilized in any form,
electronicormechanical,includingphotocopying,recording,orbyanyinformationstorageandretrievalsystem,without
writtenpermissionfromthecopyrightowner.
ProductionCredits
Publisher:CathleenSether
SeniorAcquisitionsEditor:TimothyAnderson
SeniorEditorialAssistant:StephanieSguigna
ProductionDirector:AmyRose
SeniorMarketingManager:AndreaDeFronzo
V.P.,ManufacturingandInventoryControl:ThereseConnell
Composition:NortheastCompositors,Inc.
CoverandTitlePageDesign:KristinE.Parker
CoverImage:©AlexisPuentes/ShutterStock,Inc.
PrintingandBinding:Malloy,Inc.
CoverPrinting:Malloy,Inc.
LibraryofCongressCataloging-in-PublicationData
Linz,Peter.

Anintroductiontoformallanguagesandautomata/PeterLinz.—5thed.
p.cm.
Includesbibliographicalreferencesandindex.ISBN978-1-4496-1552-9(casebound)
1.Formallanguages.2.Machinetheory.I.Title.
QA267.3.L562011
005.13’1—dc222010040050
6048
PrintedintheUnitedStatesofAmerica
151413121110987654321



TotheMemoryofmyParents



Contents
Preface
1IntroductiontotheTheoryofComputation
1.1MathematicalPreliminariesandNotation
Sets
FunctionsandRelations
GraphsandTrees
ProofTechniques
1.2ThreeBasicConcepts
Languages
Grammars
Automata
1.3SomeApplications*
2FiniteAutomata

2.1DeterministicFiniteAccepters
DeterministicAcceptersandTransitionGraphs
LanguagesandDfa’s
RegularLanguages
2.2NondeterministicFiniteAccepters
DefinitionofaNondeterministicAccepter
WhyNondeterminism?
2.3EquivalenceofDeterministicandNondeterministicFiniteAccepters
2.4ReductionoftheNumberofStatesinFiniteAutomata*
3RegularLanguagesandRegularGrammars
3.1RegularExpressions
FormalDefinitionofaRegularExpression
LanguagesAssociatedwithRegularExpressions
3.2ConnectionBetweenRegularExpressionsandRegularLanguages
RegularExpressionsDenoteRegularLanguages
RegularExpressionsforRegularLanguages


RegularExpressionsforDescribingSimplePatterns
3.3RegularGrammars
Right-andLeft-LinearGrammars
Right-LinearGrammarsGenerateRegularLanguages
Right-LinearGrammarsforRegularLanguages
EquivalenceofRegularLanguagesandRegularGrammars
4PropertiesofRegularLanguages
4.1ClosurePropertiesofRegularLanguages
ClosureunderSimpleSetOperations
ClosureunderOtherOperations
4.2ElementaryQuestionsaboutRegularLanguages
4.3IdentifyingNonregularLanguages

UsingthePigeonholePrinciple
APumpingLemma
5Context-FreeLanguages
5.1Context-FreeGrammars
ExamplesofContext-FreeLanguages
LeftmostandRightmostDerivations
DerivationTrees
RelationBetweenSententialFormsandDerivationTrees
5.2ParsingandAmbiguity
ParsingandMembership
AmbiguityinGrammarsandLanguages
5.3Context-FreeGrammarsandProgrammingLanguages
6SimplificationofContext-FreeGrammarsandNormalForms
6.1MethodsforTransformingGrammars
AUsefulSubstitutionRule
RemovingUselessProductions
Removingλ-Productions
RemovingUnit-Productions
6.2TwoImportantNormalForms
ChomskyNormalForm


GreibachNormalForm
6.3AMembershipAlgorithmforContext-FreeGrammars*
7PushdownAutomata
7.1NondeterministicPushdownAutomata
DefinitionofaPushdownAutomaton
TheLanguageAcceptedbyaPushdownAutomaton
7.2PushdownAutomataandContext-FreeLanguages
PushdownAutomataforContext-FreeLanguages

Context-FreeGrammarsforPushdownAutomata
7.3DeterministicPushdownAutomataandDeterministicContext-FreeLanguages
7.4GrammarsforDeterministicContext-FreeLanguages*
8PropertiesofContext-FreeLanguages
8.1TwoPumpingLemmas
APumpingLemmaforContext-FreeLanguages
APumpingLemmaforLinearLanguages
8.2ClosurePropertiesandDecisionAlgorithmsforContext-FreeLanguages
ClosureofContext-FreeLanguages
SomeDecidablePropertiesofContext-FreeLanguages.
9TuringMachines
9.1TheStandardTuringMachine
DefinitionofaTuringMachine
TuringMachinesasLanguageAccepters
TuringMachinesasTransducers
9.2CombiningTuringMachinesforComplicatedTasks
9.3Turing’sThesis
10OtherModelsofTuringMachines
10.1MinorVariationsontheTuringMachineTheme
EquivalenceofClassesofAutomata
TuringMachineswithaStay-Option
TuringMachineswithSemi-InfiniteTape
TheOff-LineTuringMachine
10.2TuringMachineswithMoreComplexStorage


MultitapeTuringMachines
MultidimensionalTuringMachines
10.3NondeterministicTuringMachines
10.4AUniversalTuringMachine

10.5LinearBoundedAutomata
11AHierarchyofFormalLanguagesandAutomata
11.1RecursiveandRecursivelyEnumerableLanguages
LanguagesThatAreNotRecursivelyEnumerable
ALanguageThatIsNotRecursivelyEnumerable
ALanguageThatIsRecursivelyEnumerablebutNotRecursive
11.2UnrestrictedGrammars
11.3Context-SensitiveGrammarsandLanguages
Context-SensitiveLanguagesandLinearBoundedAutomata
RelationBetweenRecursiveandContext-SensitiveLanguages
11.4TheChomskyHierarchy
12LimitsofAlgorithmicComputation
12.1SomeProblemsThatCannotBeSolvedbyTuringMachines
ComputabilityandDecidability
TheTuringMachineHaltingProblem
ReducingOneUndecidableProblemtoAnother
12.2UndecidableProblemsforRecursivelyEnumerableLanguages
12.3ThePostCorrespondenceProblem
12.4UndecidableProblemsforContext-FreeLanguages
12.5AQuestionofEfficiency
13OtherModelsofComputation
13.1RecursiveFunctions
PrimitiveRecursiveFunctions
Ackermann’sFunction
μRecursiveFunctions
13.2PostSystems
13.3RewritingSystems
MatrixGrammars

MarkovAlgorithms
L-Systems
14AnOverviewofComputationalComplexity
14.1EfficiencyofComputation
14.2TuringMachineModelsandComplexity
14.3LanguageFamiliesandComplexityClasses
14.4TheComplexityClassesPandNP
14.5SomeNPProblems
14.6Polynomial-TimeReduction
14.7NP-CompletenessandanOpenQuestion
AppendixAFinite-StateTransducers
A.1AGeneralFramework
A.2MealyMachines
A.3MooreMachines
A.4MooreandMealyMachineEquivalence
A.5MealyMachineMinimization
A.6MooreMachineMinimization
A.7LimitationsofFinite-StateTransducers
AppendixBJFLAP:ARecommendation
AnswersSolutionsandHintsforSelectedExercises
ReferencesforFurtherReading
Index



Preface
hisbookisdesignedforanintroductorycourseonformallanguages,automata,
computability, and related matters. These topics form a major part of what is
known as the theory of computation. A course on this subject matter is now
standard in the computer science curriculum and is often taught fairly early in

theprogram.Hence,theprospectiveaudienceforthisbookconsistsprimarilyof
sophomoresandjuniorsmajoringincomputerscienceorcomputerengineering.

T

Prerequisites for the material in this bookare a knowledge of some higher-level
programming language (commonly C, C++, or Java™) and familiarity with the
fundamentals of data structures and algorithms. A course in discrete mathematics that
includessettheory,functions,relations,logic,andelementsofmathematicalreasoningis
essential.Suchacourseispartofthestandardintroductorycomputersciencecurriculum.
Thestudyofthetheoryofcomputationhasseveralpurposes,mostimportantly(1)to
familiarizestudentswiththefoundationsandprinciplesofcomputerscience,(2)toteach
materialthatisusefulinsubsequentcourses,and(3)tostrengthenstudents’abilitytocarry
outformalandrigorousmathematicalarguments.ThepresentationIhavechosenforthis
textfavorsthefirsttwopurposes,althoughIwouldarguethatitalsoservesthethird.To
present ideas clearly and to give students insight into the material, the text stresses
intuitivemotivationandillustrationofideasthroughexamples.Whenthereisachoice,I
preferargumentsthatareeasilygraspedtothosethatareconciseandelegantbutdifficult
inconcept.Istatedefinitionsandtheoremspreciselyandgivethemotivationforproofs,
but often leave out the routine and tedious details. I believe that this is desirable for
pedagogical reasons. Many proofs are unexciting applications of induction or
contradiction with differences that are specific to particular problems. Presenting such
argumentsinfulldetailisnotonlyunnecessary,butinterfereswiththeflowofthestory.
Therefore,quiteafewoftheproofsarebriefandsomeonewhoinsistsoncompleteness
mayconsiderthemlackingindetail.Idonotseethisasadrawback.Mathematicalskills
arenotthebyproductofreadingsomeoneelse’sarguments,butcomefromthinkingabout
theessenceofaproblem,discoveringideassuitabletomakethepoint,thencarryingthem
outinprecisedetail.Thelatterskillcertainlyhastobelearned,andIthinkthattheproof
sketchesinthistextprovideveryappropriatestartingpointsforsuchapractice.
Computersciencestudentssometimesviewacourseinthetheoryofcomputationas

unnecessarilyabstractandofnopracticalconsequence.Toconvincethemotherwise,one
needstoappealtotheirspecificinterestsandstrengths,suchastenacityandinventiveness
indealingwithhard-to-solveproblems.Becauseofthis,myapproachemphasizeslearning
throughproblemsolving.
By a problem-solving approach, I mean that students learn the material primarily
throughproblem-typeillustrativeexamplesthatshowthemotivationbehindtheconcepts,
aswellastheirconnectiontothetheoremsanddefinitions.Atthesametime,theexamples
mayinvolveanontrivialaspect,forwhichstudentsmustdiscoverasolution.Insuchan
approach, homeworkexercises contribute to a major part of the learning process. The


exercisesattheendofeachsectionaredesignedtoilluminateandillustratethematerial
andcallonstudents’problem-solvingabilityatvariouslevels.Someoftheexercisesare
fairlysimple,pickingupwherethediscussioninthetextleavesoffandaskingstudentsto
carryonforanothersteportwo.Otherexercisesareverydifficult,challengingeventhe
best minds. The more difficult exercises are marked with a star. A good mix of such
exercises can be a very effective teaching tool. Students need not be asked to solve all
problems, but should be assigned those that support the goals of the course and the
viewpoint of the instructor. Computer science curricula differ from institution to
institution;whileafewemphasizethetheoreticalside,othersarealmostentirelyoriented
toward practical application. I believe that this text can serve either of these extremes,
provided that the exercises are selected carefully with the students’ background and
interestsinmind.Atthesametime,theinstructorneedstoinformthestudentsaboutthe
levelofabstractionthatisexpectedofthem.Thisisparticularlytrueoftheproof-oriented
exercises.WhenIsay“provethat”or“showthat,”Ihaveinmindthatthestudentshould
think about how a proof can be constructed and then produce a clear argument. How
formal such a proof should be needs to be determined by the instructor, and students
shouldbegivenguidelinesonthisearlyinthecourse.
Thecontentofthetextisappropriateforaone-semestercourse.Mostofthematerial
canbecovered,althoughsomechoiceofemphasiswillhavetobemade.Inmyclasses,I

generallyglossoverproofs,givingjustenoughcoveragetomaketheresultplausible,and
then ask students to read the rest on their own. Overall, though, little can be skipped
entirelywithoutpotentialdifficultieslateron.Afewsections,whicharemarkedwithan
asterisk, can be omitted without loss to later material. Most of the material, however, is
essentialandmustbecovered.
The fifth edition of this text introduces a substantial amount of new material. While
thepresentationinthefourtheditionhasbeenretainedwithonlyminormodifications,two
appendices have been added. The first is an entire chapter on finite-state transducers,
AppendixA. While transducers play no significant role in formal language theory, they
are important in other areas of computer science, such as digital design. Students can
benefit from an early exposure to this subject; if time permits it is worthwhile to do so.
Duetothesimilaritywithfiniteaccepters,thisinvolvesfewnewconcepts.
I also added an introduction to JFLAP, an interactive software tool that I feel is of
great help in both learning the material and in teaching this course. JFLAP implements
most of the ideas and constructions in this book. It not only helps students visualize
abstract concepts, but it is also a great time-saver. Many of the exercises in this
bookrequirecreatingstructuresthatarecomplicatedandthathavetobethoroughlytested
for correctness. JFLAP can reduce the time required for this by an order of magnitude.
Appendix B gives a brief introduction to JFLAP and the CD that comes with the
bookexpands on this. I very much recommend the use of JFLAP for both students and
instructors.

PeterLinz



Chapter1
Introductionto
theTheoryof
Computation

he subject matter of this book, the theory of computation, includes several
topics: automata theory, formal languages and grammars, computability, and
complexity. Together, this material constitutes the theoretical foundation of
computer science. Loosely speaking we can think of automata, grammars, and
computabilityasthestudyofwhatcanbedonebycomputersinprinciple,while
complexityaddresseswhatcanbedoneinpractice.Inthisbookwefocusalmostentirely
onthefirstoftheseconcerns.Wewillstudyvariousautomata,seehowtheyarerelatedto
languages and grammars, and investigate what can and cannot be done by digital
computers.Althoughthistheoryhasmanyuses,itisinherentlyabstractandmathematical.

T

Computerscienceisapracticaldiscipline.Thosewhoworkinitoftenhaveamarked
preferenceforusefulandtangibleproblemsovertheoreticalspeculation.Thisiscertainly
true of computer science students who are concerned mainly with difficult applications
fromtherealworld.Theoreticalquestionsinterestthemonlyiftheyhelpinfindinggood
solutions. This attitude is appropriate, since without applications there would be little
interestincomputers.Butgiventhispracticalorientation,onemightwellask“whystudy
theory?”
Thefirstansweristhattheoryprovidesconceptsandprinciplesthathelpusunderstand
thegeneralnatureofthediscipline.Thefieldofcomputerscienceincludesawiderangeof
special topics, from machine design to programming. The use of computers in the real
worldinvolvesawealthofspecificdetailthatmustbelearnedforasuccessfulapplication.
This makes computer science a very diverse and broad discipline. But in spite of this
diversity,therearesomecommonunderlyingprinciples.Tostudythesebasicprinciples,
we construct abstract models of computers and computation. These models embody the
importantfeaturesthatarecommontobothhardwareandsoftwareandthatareessentialto
manyofthespecialandcomplexconstructsweencounterwhileworkingwithcomputers.
Even when such models are too simple to be applicable immediately to real-world
situations, the insights we gain from studying them provide the foundation on which

specific development is based. This approach is, of course, not unique to computer
science. The construction of models is one of the essentials of any scientific discipline,
and the usefulness of a discipline is often dependent on the existence of simple, yet
powerful,theoriesandlaws.
Asecond,andperhapsnotsoobvious,answeristhattheideaswewilldiscusshave
some immediate and important applications. The fields of digital design, programming
languages,andcompilersarethemostobviousexamples,buttherearemanyothers.The
concepts we study here run like a thread through much of computer science, from


operatingsystemstopatternrecognition.
Thethirdanswerisoneofwhichwehopetoconvincethereader.Thesubjectmatteris
intellectuallystimulatingandfun.Itprovidesmanychallenging,puzzle-likeproblemsthat
canleadtosomesleeplessnights.Thisisproblemsolvinginitspureessence.
Inthisbook,wewilllookatmodelsthatrepresentfeaturesatthecoreofallcomputers
andtheirapplications.Tomodelthehardwareofacomputer,weintroducethenotionofan
automaton (plural, automata). An automaton is a construct that possesses all the
indispensablefeaturesofadigitalcomputer.Itacceptsinput,producesoutput,mayhave
sometemporarystorage,andcanmakedecisionsintransformingtheinputintotheoutput.
A formal language is an abstraction of the general characteristics of programming
languages.Aformallanguageconsistsofasetofsymbolsandsomerulesofformationby
whichthesesymbolscanbecombinedintoentitiescalledsentences.Aformallanguageis
thesetofallsentencespermittedbytherulesofformation.Althoughsomeoftheformal
languageswestudyherearesimplerthanprogramminglanguages,theyhavemanyofthe
same essential features. We can learn a great deal about programming languages from
formallanguages.Finally,wewillformalizetheconceptofamechanicalcomputationby
givingaprecisedefinitionofthetermalgorithmandstudythekindsofproblemsthatare
(andarenot)suitableforsolutionbysuchmechanicalmeans.Inthecourseofourstudy,
we will show the close connection between these abstractions and investigate the
conclusionswecanderivefromthem.

Inthefirstchapter,welookatthesebasicideasinaverybroadwaytosetthestagefor
later work. In Section 1.1, we review the main ideas from mathematics that will be
required.Whileintuitionwillfrequentlybeourguideinexploringideas,theconclusions
we draw will be based on rigorous arguments. This will involve some mathematical
machinery,althoughtherequirementsarenotextensive.Thereaderwillneedareasonably
goodgraspoftheterminologyandoftheelementaryresultsofsettheory,functions,and
relations. Trees and graph structures will be used frequently, although little is needed
beyondthedefinitionofalabeled,directedgraph.Perhapsthemoststringentrequirement
is the ability to follow proofs and an understanding of what constitutes proper
mathematical reasoning. This includes familiarity with the basic proof techniques of
deduction,induction,andproofbycontradiction.Wewillassumethatthereaderhasthis
necessarybackground.Section1.1isincludedtoreviewsomeofthemainresultsthatwill
beusedandtoestablishanotationalcommongroundforsubsequentdiscussion.
InSection1.2,wetakeafirstlookatthecentralconceptsoflanguages,grammars,and
automata.Theseconceptsoccurinmanyspecificformsthroughoutthebook.InSection
1.3, we give some simple applications of these general ideas to illustrate that these
conceptshavewidespreadusesincomputerscience.Thediscussioninthesetwosections
willbeintuitiveratherthanrigorous.Later,wewillmakeallofthismuchmoreprecise;
butforthemoment,thegoalistogetaclearpictureoftheconceptswithwhichweare
dealing.

1.1 MathematicalPreliminariesandNotation
Sets


Asetisacollectionofelements,withoutanystructureotherthanmembership.Toindicate
thatxisanelementofthesetS,wewritex∈S.ThestatementthatxisnotinSiswrittenx
∉S.Asetcanbespecifiedbyenclosingsomedescriptionofitselementsincurlybraces;
forexample,thesetofintegers0,1,2isshownas
S={0,1,2}.

Ellipses are used whenever the meaning is clear. Thus, {a, b,…, z} stands for all the
lowercaselettersoftheEnglishalphabet,while{2,4,6,…}denotesthesetofallpositive
evenintegers.Whentheneedarises,weusemoreexplicitnotation,inwhichwewrite
forthelastexample.Wereadthisas“Sisthesetofalli,suchthatiisgreaterthanzero,
andiiseven,”implying,ofcourse,thatiisaninteger.
Theusualsetoperationsareunion(∪),intersection(∩),anddifference(−)defined
as

Anotherbasicoperationiscomplementation.ThecomplementofasetS,denotedby
consistsofallelementsnotinS.Tomakethismeaningful,weneedtoknowwhatthe
universalsetUofallpossibleelementsis.IfUisspecified,then

Thesetwithnoelements,calledtheemptysetorthenullset,isdenotedby∅.From
thedefinitionofaset,itisobviousthat

Thefollowingusefulidentities,knownasDeMorgan’slaws,

areneededonseveraloccasions.
AsetS1issaidtobeasubsetofSifeveryelementofS1isalsoanelementofS.We
writethisas
S1⊆S.
IfS1⊆S,butScontainsanelementnotinS1,wesaythatS1isapropersubsetofS;we
writethisas
S1⊂S.


IfS1andS2 have no common element, that is, S1∩S2 = ø, then the sets are said to be
disjoint.
A set is said to be finite if it contains a finite number of elements; otherwise it is
infinite.Thesizeofafinitesetisthenumberofelementsinit;thisisdenotedby|S|.

Agivensetnormallyhasmanysubsets.ThesetofallsubsetsofasetSiscalledthe
powersetofSandisdenotedby2s.Observethat2sisasetofsets.
Example1.1
IfSistheset{a,b,c},thenitspowersetis
Here|S|=3and|2s|=8.Thisisaninstanceofageneralresult;ifSisfinite,then

Inmanyofourexamples,theelementsofasetareorderedsequencesofelementsfrom
othersets.SuchsetsaresaidtobetheCartesianproductofothersets.FortheCartesian
productoftwosets,whichitselfisasetoforderedpairs,wewrite

Example1.2
LetS1={2,4}andS2={2,3,5,6}.Then
S1×S2={(2,2),(2,3),(2,5),(2,6),(4,2),(4,3),(4,5),(4,6)}.
Notethattheorderinwhichtheelementsofapairarewrittenmatters.Thepair(4,2)isin
S1×S2,but(2,4)isnot.
ThenotationisextendedinanobviousfashiontotheCartesianproductofmorethan
twosets;generally

Asetcanbedividedbyseparatingitintoanumberofsubsets.SupposethatS1,S2,Sn
aresubsetsofagivensetSandthatthefollowingholds:
1.ThesubsetsS1,S2,…Snaremutuallydisjoint;
2.S1∪S2∪…∪Sn=S;
3.noneoftheSiisempty.
ThenS1,S2,…SniscalledapartitionofS.

FunctionsandRelations
Afunctionisarulethatassignstoelementsofonesetauniqueelementofanotherset.Iff
denotes a function, then the first set is called the domainoff, and the second set is its
range.Wewrite

f:S1→S2
toindicatethatthedomainoffisasubsetofS1andthattherangeoffisasubsetofS2.If
thedomainoffisallofS1,wesaythatfisatotalfunctiononS1;otherwisefissaidtobe
apartialfunction.
Inmanyapplications,thedomainandrangeofthefunctionsinvolvedareinthesetof
positive integers. Furthermore, we are often interested only in the behavior of these
functions as their arguments become very large. In such cases an understanding of the
growthratesmaysufficeandacommonorderofmagnitudenotationcanbeused.Letf(n)
andg(n)befunctionswhosedomainisasubsetofthepositiveintegers.Ifthereexistsa
positiveconstantcsuchthatforallsufficientlylargen
wesaythatfhasorderatmostg.Wewritethisas
If
thenfhasorderatleastg,forwhichweuse
Finally,ifthereexistconstantsc1andc2suchthat
fandghavethesameorderofmagnitude,expressedas
In this order-of-magnitude notation, we ignore multiplicative constants and lower-order
termsthatbecomenegligibleasnincreases.
Example1.3
Let

Then

Inorder-of-magnitudenotation,thesymbol=shouldnotbeinterpretedasequalityand
order-of-magnitude expressions cannot be treated like ordinary expressions.
Manipulationssuchas


arenotsensibleandcanleadtoincorrectconclusions.Still,ifusedproperly,theorder-ofmagnitudeargumentscanbeeffective,aswewillseeinlaterchapters.
Somefunctionscanberepresentedbyasetofpairs

wherexiisanelementinthedomainofthefunction,andyiisthecorrespondingvaluein
itsrange.Forsuchasettodefineafunction,eachxicanoccuratmostonceasthefirst
element of a pair. If this is not satisfied, the set is called a relation.Relationsaremore
general than functions: In a function each element of the domain has exactly one
associated element in the range; in a relation there may be several such elements in the
range.
Onekindofrelationisthatofequivalence,ageneralizationoftheconceptofequality
(identity).Toindicatethatapair(x,y)isinanequivalencerelation,wewrite
x≡y.
A relation denoted by ≡ is considered an equivalence if it satisfies three rules: the
reflexivityrule
thesymmetryrule
andthetransitivityrule
Example1.4
Onthesetofnonnegativeintegers,wecandefinearelation
ifandonlyif
Then 2 ≡ 5, 12 ≡ 0, and 0 ≡ 36. Clearly this is an equivalence relation, as it satisfies
reflexivity,symmetry,andtransitivity.
IfS is a set on which we have a defined equivalence relation, then we can use this
equivalencetopartitionthesetintoequivalenceclasses.Eachequivalenceclasscontains
allandonlyequivalentelements.

GraphsandTrees
Agraphisaconstructconsistingoftwofinitesets,thesetV={υ1,υ2,…,υn}ofvertices
and the set E = {e1, e2,…, em} of edges. Each edge is a pair of vertices from V, for
instance,


isanedgefromυjtoυk.Wesaythattheedgeeiisanoutgoingedgeforυjandanincoming
edgeforυk.Suchaconstructisactuallyadirectedgraph(digraph),sinceweassociatea

direction(fromυjtoυk)witheachedge.Graphsmaybelabeled,alabelbeinganameor
other information associated with parts of the graph. Both vertices and edges may be
labeled.
Graphsareconvenientlyvisualizedbydiagramsinwhichtheverticesarerepresented
as circles and the edges as lines with arrows connecting the vertices. The graph with
vertices{υ1,υ2,υ3}andedges{(υ1,υ3),(υ3,υ1),(υ3,υ2),(υ3,υ3)}isdepictedinFigure1.1.
Asequenceofedges(υi,υj),(υj,υk),…,(υm,υn)issaidtobeawalkfromυitoυn.The
lengthofawalkisthetotalnumberofedgestraversedingoingfromtheinitialvertexto
thefinalone.Awalkinwhichnoedgeisrepeatedissaidtobeapath;apathissimpleif
novertexisrepeated.Awalkfromυitoitselfwithnorepeatededgesiscalledacyclewith
baseυi. If no vertices other than the base are repeated in a cycle, then it is said to be
simple. In Figure1.1,(υ1,υ3),(υ3,υ2) is a simple path from υ1 to υ2. The sequence of
edges(υ1,υ3),(υ3,υ3),(υ3,υ1)isacycle,butnotasimpleone.Iftheedgesofagraphare
labeled,wecantalkaboutthelabelofawalk.Thislabelisthesequenceofedgelabels
encounteredwhenthepathistraversed.Finally,anedgefromavertextoitselfiscalleda
loop.InFigure1.1,thereisalooponvertexυ3.
Figure1.1

On several occasions, we will refer to an algorithm for finding all simple paths
betweentwogivenvertices(orallsimplecyclesbasedonavertex).Ifwedonotconcern
ourselves with efficiency, we can use the following obvious method. Starting from the
given vertex, say υi, list all outgoing edges (υi,υk),(υi, υl),…At this point, we have all
pathsoflengthonestartingatυi.Forallverticesυk,υl,…soreached,welistalloutgoing
edges as long as they do not lead to any vertex already used in the path we are
constructing.Afterwedothis,wewillhaveallsimplepathsoflengthtwooriginatingat
υi.Wecontinuethisuntilallpossibilitiesareaccountedfor.Sincethereareonlyafinite
numberofvertices,wewilleventuallylistallsimplepathsbeginningatυi.Fromthesewe
selectthoseendingatthedesiredvertex.
Treesareaparticulartypeofgraph.Atreeisadirectedgraphthathasnocycles,and
thathasonedistinctvertex,calledtheroot,suchthatthereisexactlyonepathfromthe

roottoeveryothervertex.Thisdefinitionimpliesthattheroothasnoincomingedgesand
that there are some vertices without outgoing edges. These are called the leaves of the
tree.Ifthereisanedgefromυitoυj,thenυiissaidtobetheparentofυj,andυjthechild
ofυi.Thelevelassociatedwitheachvertexisthenumberofedgesinthepathfromthe


roottothevertex.Theheightofthetreeisthelargestlevelnumberofanyvertex.These
termsareillustratedinFigure1.2.
Attimes,wewanttoassociateanorderingwiththenodesateachlevel;insuchcases
wetalkaboutorderedtrees.
Figure1.2

Moredetailsongraphsandtreescanbefoundinmostbooksondiscretemathematics.

ProofTechniques
An important requirement for reading this text is the ability to follow proofs. In
mathematicalarguments,weemploytheacceptedrulesofdeductivereasoning,andmany
proofs are simply a sequence of such steps. Two special proof techniques are used so
frequentlythatitisappropriatetoreviewthembriefly.Theseareproofbyinductionand
proofbycontradiction.
Inductionisatechniquebywhichthetruthofanumberofstatementscanbeinferred
fromthetruthofafewspecificinstances.SupposewehaveasequenceofstatementsP1,
P2,…wewanttoprovetobetrue.Furthermore,supposealsothatthefollowingholds:
1.Forsomek≥1,weknowthatP1,P2,…,Pkaretrue.
2.Theproblemissuchthatforanyn≥k,thetruthsofP1,P2,…,Pnimplythetruthof
Pn+1.
Wecanthenuseinductiontoshowthateverystatementinthissequenceistrue.
Inaproofbyinduction,weargueasfollows:FromCondition1weknowthatthefirst
kstatementsaretrue.ThenCondition2tellsusthatPk+1alsomustbetrue.Butnowthat
weknowthatthefirstk+1statementsaretrue,wecanapplyCondition2againtoclaim

thatPk+2mustbetrue,andsoon.Weneednotexplicitlycontinuethisargument,because
thepatternisclear.Thechainofreasoningcanbeextendedtoanystatement.Therefore,
everystatementistrue.
The starting statements P1, P2,…Pk are called the basis of the induction. The step


connectingPnwithPn+1iscalledtheinductivestep.Theinductivestepisgenerallymade
easierbytheinductiveassumptionthatP1,P2,…,Pnaretrue,thenarguethatthetruthof
thesestatementsguaranteesthetruthofPn+1.Inaformalinductiveargument,weshowall
threepartsexplicitly.
Example1.5
Abinarytreeisatreeinwhichnoparentcanhavemorethantwochildren.Provethata
binarytreeofheightnhasatmost2nleaves.
Proof:Ifwedenotethemaximumnumberofleavesofabinarytreeofheightnbyl(n),
thenwewanttoshowthatl(n)≤2n.
Basis:Clearlyl(0)=1=20sinceatreeofheight0canhavenonodesotherthantheroot,
thatis,ithasatmostoneleaf.
InductiveAssumption:
InductiveStep:Togetabinarytreeofheightn+1fromoneofheightn,wecancreate,at
most,twoleavesinplaceofeachpreviousone.Therefore,
Now,usingtheinductiveassumption,weget

Thus,ifourclaimistrueforn,itmustalsobetrueforn+1.Sincencanbeanynumber,
thestatementmustbetrueforalln.
Hereweintroducethesymbol thatisusedinthisbooktodenotetheendofaproof.
Inductive reasoning can be difficult to grasp. It helps to notice the close connection
betweeninductionandrecursioninprogramming.Forexample,therecursivedefinitionof
a function f(n), where n is any positive integer, often has two parts. One involves the
definitionoff(n+1)intermsoff(n),f(n−1),…,f(1).Thiscorrespondstotheinductive
step. The second part is the “escape” from the recursion, which is accomplished by

definingf(1),f(2),…,f(k)nonrecursively.Thiscorrespondstothebasisofinduction.As
ininduction,recursionallowsustodrawconclusionsaboutallinstancesoftheproblem,
givenonlyafewstartingvaluesandusingtherecursivenatureoftheproblem.
Sometimes, a problem looks difficult until we look at it in just the right way. Often
lookingatitrecursivelysimplifiesmattersgreatly.
Example1.6
Asetl1,l2,…,lnofmutuallyintersectingstraightlinesdividestheplaneintoanumberof
separatedregions.Asinglelinedividestheplaneintotwoparts,twolinesgeneratefour
regions,threelinesmakesevenregions,andsoon.Thisiseasilycheckedvisuallyforup
tothreelines,butasthenumberoflinesincreasesitbecomesdifficulttospotapattern.
Letustrytosolvethisproblemrecursively.


LookatFigure1.3toseewhathappensifweaddanewlineln+1toexistingnlines.
Theregiontotheleftofl1isdividedintotwonewregions,soistheregiontotheleftofl2,
andsoonuntilwegettothelastline.Atthelastline,theregiontotherightoflnisalso
divided.Eachofthenintersectionsthengeneratesonenewregion,withoneextraatthe
end.So,
Figure1.3

ifweletA(n)denotethenumberofregionsgeneratedbynlines,weseethat

withA(1)=2.FromthissimplerecursionwethencalculateA(2)=4,A(3)=7,A(4)=
11,andsoon.
To get a formula for A (n) and to show that it is correct, we use induction. If we
conjecturethat

then

justifiestheinductivestep.Thebasisiseasilychecked,completingtheargument.

In this example we have been a little less formal in identifying the basis, inductive
assumption, and inductive step, but they are there and are essential. To keep our
subsequent discussions from becoming too formal, we will generally prefer the style of
thissecondexample.However,ifyouhavedifficultyinfollowingorconstructingaproof,
gobacktothemoreexplicitformofExample1.5.
Proofbycontradictionisanotherpowerfultechniquethatoftenworkswheneverything
elsefails.SupposewewanttoprovethatsomestatementPistrue.Wethenassume,for
the moment, that P is false and see where that assumption leads us. If we arrive at a
conclusionthatweknowisincorrect,wecanlaytheblameonthestartingassumptionand
concludethatPmustbetrue.Thefollowingisaclassicandelegantexample.
Example1.7