Operating systems principles and practice 2nd by thomas anderson

OperatingSystems
Principles&Practice
VolumeI:KernelsandProcesses
SecondEdition
ThomasAnderson
UniversityofWashington

MikeDahlin
UniversityofTexasandGoogle
RecursiveBooks
recursivebooks.com


OperatingSystems:PrinciplesandPractice(SecondEdition)VolumeI:Kernelsand
ProcessesbyThomasAndersonandMichaelDahlin
Copyright©ThomasAndersonandMichaelDahlin,2011-2015.
ISBN978-0-9856735-3-6
Publisher:RecursiveBooks,Ltd.,
Cover:ReflectionLake,Mt.Rainier
Coverdesign:CameronNeat
Illustrations:CameronNeat
Copyeditors:SandyKaplan,WhitneySchmidt
Ebookdesign:RobinBriggs
Webdesign:AdamAnderson
SUGGESTIONS,COMMENTS,andERRORS.Wewelcomesuggestions,commentsand
errorreports,byemailto
Noticeofrights.Allrightsreserved.Nopartofthisbookmaybereproduced,storedina
retrievalsystem,ortransmittedinanyformbyanymeans—electronic,mechanical,
photocopying,recording,orotherwise—withoutthepriorwrittenpermissionofthe
publisher.Forinformationongettingpermissionsforreprintsandexcerpts,contact

Noticeofliability.Theinformationinthisbookisdistributedonan“AsIs”basis,without
warranty.NeithertheauthorsnorRecursiveBooksshallhaveanyliabilitytoanypersonor
entitywithrespecttoanylossordamagecausedorallegedtobecauseddirectlyor
indirectlybytheinformationorinstructionscontainedinthisbookorbythecomputer
softwareandhardwareproductsdescribedinit.
Trademarks:Throughoutthisbooktrademarkednamesareused.Ratherthanputa
trademarksymbolineveryoccurrenceofatrademarkedname,westateweareusingthe
namesonlyinaneditorialfashionandtothebenefitofthetrademarkownerwithno
intentionofinfringementofthetrademark.Alltrademarksorservicemarksarethe
propertyoftheirrespectiveowners.


ToRobin,Sandra,Katya,andAdam
TomAnderson
ToMarla,Kelly,andKeith
MikeDahlin



Contents
Preface

IKernelsandProcesses
1Introduction
1.1WhatIsAnOperatingSystem?
1.1.1ResourceSharing:OperatingSystemasReferee
1.1.2MaskingLimitations:OperatingSystemasIllusionist
1.1.3ProvidingCommonServices:OperatingSystemasGlue
1.1.4OperatingSystemDesignPatterns

1.2OperatingSystemEvaluation
1.2.1ReliabilityandAvailability
1.2.2Security
1.2.3Portability
1.2.4Performance
1.2.5Adoption
1.2.6DesignTradeoffs
1.3OperatingSystems:Past,Present,andFuture
1.3.1ImpactofTechnologyTrends
1.3.2EarlyOperatingSystems
1.3.3Multi-UserOperatingSystems
1.3.4Time-SharingOperatingSystems
1.3.5ModernOperatingSystems
1.3.6FutureOperatingSystems
Exercises
2TheKernelAbstraction
2.1TheProcessAbstraction
2.2Dual-ModeOperation
2.2.1PrivilegedInstructions
2.2.2MemoryProtection
2.2.3TimerInterrupts
2.3TypesofModeTransfer
2.3.1UsertoKernelMode
2.3.2KerneltoUserMode
2.4ImplementingSafeModeTransfer
2.4.1InterruptVectorTable
2.4.2InterruptStack


2.4.3TwoStacksperProcess

2.4.4InterruptMasking
2.4.5HardwareSupportforSavingandRestoringRegisters
2.5PuttingItAllTogether:x86ModeTransfer
2.6ImplementingSecureSystemCalls
2.7StartingaNewProcess
2.8ImplementingUpcalls
2.9CaseStudy:BootinganOperatingSystemKernel
2.10CaseStudy:VirtualMachines
2.11SummaryandFutureDirections
Exercises
3TheProgrammingInterface
3.1ProcessManagement
3.1.1WindowsProcessManagement
3.1.2UNIXProcessManagement
3.2Input/Output
3.3CaseStudy:ImplementingaShell
3.4CaseStudy:InterprocessCommunication
3.4.1Producer-ConsumerCommunication
3.4.2Client-ServerCommunication
3.5OperatingSystemStructure
3.5.1MonolithicKernels
3.5.2Microkernel
3.6SummaryandFutureDirections
Exercises

II:Concurrency
4.ConcurrencyandThreads
5.SynchronizingAccesstoSharedObjects
6.Multi-ObjectSynchronization
7.Scheduling

III:MemoryManagement
8.AddressTranslation


9.CachingandVirtualMemory
10.AdvancedMemoryManagement

IV:PersistentStorage
11.FileSystems:IntroductionandOverview
12.StorageDevices
13.FilesandDirectories
14.ReliableStorage
References
Glossary
AbouttheAuthors



Preface
PrefacetotheeBookEdition
OperatingSystems:PrinciplesandPracticeisatextbookforafirstcoursein
undergraduateoperatingsystems.Inuseatover50collegesanduniversitiesworldwide,
thistextbookprovides:
Apathforstudentstounderstandhighlevelconceptsallthewaydowntoworking
code.
Extensiveworkedexamplesintegratedthroughoutthetextprovidestudentsconcrete
guidanceforcompletinghomeworkassignments.
Afocusonup-to-dateindustrytechnologiesandpractice
TheeBookeditionissplitintofourvolumesthattogethercontainexactlythesame

materialasthe(2nd)printeditionofOperatingSystems:PrinciplesandPractice,
reformattedforvariousscreensizes.Eachvolumeisself-containedandcanbeusedasa
standalonetext,e.g.,atschoolsthatteachoperatingsystemstopicsacrossmultiple
courses.
Volume1:KernelsandProcesses.ThisvolumecontainsChapters1-3oftheprint
edition.Wedescribetheessentialstepsneededtoisolateprogramstopreventbuggy
applicationsandcomputervirusesfromcrashingortakingcontrolofyoursystem.
Volume2:Concurrency.ThisvolumecontainsChapters4-7oftheprintedition.We
provideaconcretemethodologyforwritingcorrectconcurrentprogramsthatisin
widespreaduseinindustry,andweexplainthemechanismsforcontextswitchingand
synchronizationfromfundamentalconceptsdowntoassemblycode.
Volume3:MemoryManagement.ThisvolumecontainsChapters8-10oftheprint
edition.Weexplainboththetheoryandmechanismsbehind64-bitaddressspace
translation,demandpaging,andvirtualmachines.
Volume4:PersistentStorage.ThisvolumecontainsChapters11-14oftheprint
edition.Weexplainthetechnologiesunderlyingmodernextent-based,journaling,and
versioningfilesystems.
Amoredetaileddescriptionofeachchapterisgivenintheprefacetotheprintedition.

PrefacetothePrintEdition
WhyWeWroteThisBook
Manyofourstudentstellusthatoperatingsystemswasthebestcoursetheytookasan
undergraduateandalsothemostimportantfortheircareers.Wearenotalone—manyof
ourcolleaguesreportreceivingsimilarfeedbackfromtheirstudents.
Partoftheexcitementisthatthecoreideasinamodernoperatingsystem—protection,
concurrency,virtualization,resourceallocation,andreliablestorage—havebecome


widelyappliedthroughoutcomputerscience,notjustoperatingsystemkernels.Whether
yougetajobatFacebook,Google,Microsoft,oranyotherleading-edgetechnology

company,itisimpossibletobuildresilient,secure,andflexiblecomputersystemswithout
theabilitytoapplyoperatingsystemsconceptsinavarietyofsettings.Inamodernworld,
nearlyeverythingauserdoesisdistributed,nearlyeverycomputerismulti-core,security
threatsabound,andmanyapplicationssuchaswebbrowsershavebecomemini-operating
systemsintheirownright.
Itshouldbenosurprisethatformanycomputersciencestudents,anundergraduate
operatingsystemsclasshasbecomeadefactorequirement:atickettoaninternshipand
eventuallytoafull-timeposition.
Unfortunately,manyoperatingsystemstextbooksarestillstuckinthepast,failingtokeep
pacewithrapidtechnologicalchange.Severalwidely-usedbookswereinitiallywrittenin
themid-1980’s,andtheyoftenactasiftechnologystoppedatthatpoint.Evenwhennew
topicsareadded,theyaretreatedasanafterthought,withoutpruningmaterialthathas
becomelessimportant.Theresultaretextbooksthatareverylong,veryexpensive,andyet
failtoprovidestudentsmorethanasuperficialunderstandingofthematerial.
Ourviewisthatoperatingsystemshavechangeddramaticallyoverthepasttwentyyears,
andthatjustifiesafreshlookatbothhowthematerialistaughtandwhatistaught.The
paceofinnovationinoperatingsystemshas,ifanything,increasedoverthepastfewyears,
withtheintroductionoftheiOSandAndroidoperatingsystemsforsmartphones,theshift
tomulticorecomputers,andtheadventofcloudcomputing.
Topreparestudentsforthisnewworld,webelievestudentsneedthreethingstosucceedat
understandingoperatingsystemsatadeeplevel:
Conceptsandcode.Webelieveitisimportanttoteachstudentsbothprinciplesand
practice,conceptsandimplementation,ratherthaneitheralone.Thistextbooktakes
conceptsallthewaydowntothelevelofworkingcode,e.g.,howacontextswitch
worksinassemblycode.Inourexperience,thisistheonlywaystudentswillreally
understandandmasterthematerial.Allofthecodeinthisbookisavailablefromthe
author’swebsite,ospp.washington.edu.
Extensiveworkedexamples.Inourview,studentsneedtobeabletoapplyconcepts
inpractice.Tothatend,wehaveintegratedalargenumberofexampleexercises,
alongwithsolutions,throughoutthetext.Weusestheseexercisesextensivelyinour

ownlectures,andwehavefoundthemessentialtochallengingstudentstogobeyond
asuperficialunderstanding.
Industrypractice.Toshowstudentshowtoapplyoperatingsystemsconceptsina
varietyofsettings,weusedetailed,concreteexamplesfromFacebook,Google,
Microsoft,Apple,andotherleading-edgetechnologycompaniesthroughoutthe
textbook.Becauseoperatingsystemsconceptsareimportantinawiderangeof
computersystems,wetaketheseexamplesnotonlyfromtraditionaloperating
systemslikeLinux,Windows,andOSXbutalsofromothersystemsthatneedto
solveproblemsofprotection,concurrency,virtualization,resourceallocation,and
reliablestoragelikedatabases,webbrowsers,webservers,mobileapplications,and
searchengines.


Takingafreshperspectiveonwhatstudentsneedtoknowtoapplyoperatingsystems
conceptsinpracticehasledustoinnovateineverymajortopiccoveredinan
undergraduate-levelcourse:
KernelsandProcesses.Thesafeexecutionofuntrustedcodehasbecomecentralto
manytypesofcomputersystems,fromwebbrowserstovirtualmachinestooperating
systems.YetexistingtextbookstreatprotectionasasideeffectofUNIXprocesses,as
iftheyaresynonyms.Instead,westartfromfirstprinciples:whataretheminimum
requirementsforprocessisolation,howcansystemsimplementprocessisolation
efficiently,andwhatdostudentsneedtoknowtoimplementfunctionscorrectlywhen
thecallerispotentiallymalicious?
Concurrency.Withtheadventofmulti-corearchitectures,moststudentstodaywill
spendmuchoftheircareerswritingconcurrentcode.Existingtextbooksprovidea
blizzardofconcurrencyalternatives,mostofwhichwereabandoneddecadesagoas
impractical.Instead,wefocusonprovidingstudentsasinglemethodologybasedon
Mesamonitorsthatwillenablestudentstowritecorrectconcurrentprograms—a
methodologythatisbyfarthedominantapproachusedinindustry.
MemoryManagement.Evenasdemand-paginghasbecomelessimportant,

virtualizationhasbecomeevenmoreimportanttomoderncomputersystems.We
provideadeeptreatmentofaddresstranslationhardware,sparseaddressspaces,
TLBs,andon-chipcaches.Wethenusethoseconceptsasaspringboardfor
describingvirtualmachinesandrelatedconceptssuchascheckpointingandcopy-onwrite.
PersistentStorage.Reliablestorageinthepresenceoffailuresiscentraltothe
designofmostcomputersystems.Existingtextbookssurveythehistoryoffile
systems,spendingmostoftheirtimeadhocapproachestofailurerecoveryanddefragmentation.Yetnomodernfilesystemsstillusethoseadhocapproaches.Instead,
ourfocusisonhowfilesystemsuseextents,journaling,copy-on-write,andRAIDto
achievebothhighperformanceandhighreliability.

IntendedAudience
OperatingSystems:PrinciplesandPracticeisatextbookforafirstcoursein
undergraduateoperatingsystems.Webelieveoperatingsystemsshouldbetakenasearly
aspossibleinanundergraduate’scourseofstudy;manystudentsusethecourseasa
springboardtoaninternshipandacareer.Tothatend,wehavedesignedthetextbookto
assumeminimalpre-requisites:specifically,studentsshouldhavetakenadatastructures
courseandoneoncomputerorganization.Thecodeexamplesarewritteninacombination
ofx86assembly,C,andC++.Inparticular,wehavedesignedthebooktointerfacewell
withtheBryantandO’Hallorantextbook.Wereviewandcoverinmuchmoredepththe
materialfromthesecondhalfofthatbook.
Weshouldnotewhatthistextbookisnot:itisnotintendedtoteachtheAPIorinternalsof
anyspecificoperatingsystem,suchasLinux,Android,Windows8,OSX,oriOS.Weuse
manyconcreteexamplesfromthesesystems,butourfocusisonthesharedproblemsthese


systemsfaceandthetechnologiesthesesystemsusetosolvethoseproblems.

AGuidetoInstructors
Oneofourgoalsisenableinstructorstochooseanappropriatelevelofdepthforeach
coursetopic.Eachchapterbeginsataconceptuallevel,withimplementationdetailsand

themoreadvancedmaterialtowardstheend.Themoreadvancedmaterialcanbeomitted
withoutcompromisingtheabilityofstudentstofollowlatermaterial.Nosingle-quarteror
single-semestercourseislikelytobeabletocovereverytopicwehaveincluded,butwe
thinkitisagoodthingforstudentstocomeawayfromanoperatingsystemscoursewith
anappreciationthatthereisalwaysmoretolearn.
Foreachtopic,weattempttoconveyitatthreelevels:
Howtoreasonaboutsystems.Wedescribecoresystemsconcepts,suchas
protection,concurrency,resourcescheduling,virtualization,andstorage,andwe
providepracticeapplyingtheseconceptsinvarioussituations.Inourview,this
providesthebiggestlong-termpayofftostudents,astheyarelikelytoneedtoapply
theseconceptsintheirworkthroughouttheircareer,almostregardlessofwhat
projecttheyendupworkingon.
Powertools.Weintroducestudentstoanumberofabstractionsthattheycanapplyin
theirworkinindustryimmediatelyaftergraduation,andthatweexpectwillcontinue
tobeusefulfordecadessuchassandboxing,protectedprocedurecalls,threads,locks,
conditionvariables,caching,checkpointing,andtransactions.
Detailsofspecificoperatingsystems.Weincludenumerousexamplesofhow
differentoperatingsystemsworkinpractice.However,thismaterialchangesrapidly,
andthereisanorderofmagnitudemorematerialthancanbecoveredinasingle
semester-lengthcourse.Thepurposeoftheseexamplesistoillustratehowtousethe
operatingsystemsprinciplesandpowertoolstosolveconcreteproblems.Wedonot
attempttoprovideacomprehensivedescriptionofLinux,OSX,oranyother
particularoperatingsystem.
Thebookisdividedintofiveparts:anintroduction(Chapter1),kernelsandprocesses
(Chapters2-3),concurrency,synchronization,andscheduling(Chapters4-7),memory
management(Chapters8-10),andpersistentstorage(Chapters11-14).
Introduction.ThegoalofChapter1istointroducetherecurringthemesfoundinthe
laterchapters.Wedefinesomecommonterms,andweprovideabitofthehistoryof
thedevelopmentofoperatingsystems.
TheKernelAbstraction.Chapter2coverskernel-basedprocessprotection—the

conceptandimplementationofexecutingauserprogramwithrestrictedprivileges.
Giventheincreasingimportanceofcomputersecurityissues,webelieveprotected
executionandsafetransferacrossprivilegelevelsareworthtreatingindepth.We
havebrokenthedescriptionintosections,toallowinstructorstochooseeitheraquick
introductiontotheconcepts(upthroughSection2.3),orafulltreatmentofthekernel
implementationdetailsdowntothelevelofinterrupthandlers.Someinstructorsstart


withconcurrency,andcoverkernelsandkernelprotectionafterwards.Whileour
textbookcanbeusedthatway,wehavefoundthatstudentsbenefitfromabasic
understandingoftheroleofoperatingsystemsinexecutinguserprograms,before
introducingconcurrency.
TheProgrammingInterface.Chapter3isintendedasanimpedancematchfor
studentsofdifferingbackgrounds.Dependingonstudentbackground,itcanbe
skippedorcoveredindepth.Thechaptercoverstheoperatingsystemfroma
programmer’sperspective:processcreationandmanagement,device-independent
input/output,interprocesscommunication,andnetworksockets.Ourgoalisthat
studentsshouldunderstandatadetailedlevelwhathappenswhenauserclicksalink
inawebbrowser,astherequestistransferredthroughoperatingsystemkernelsand
userspaceprocessesattheclient,server,andbackagain.Thischapteralsocoversthe
organizationoftheoperatingsystemitself:howdevicedriversandthehardware
abstractionlayerworkinamodernoperatingsystem;thedifferencebetweena
monolithicandamicrokerneloperatingsystem;andhowpolicyandmechanismare
separatedinmodernoperatingsystems.
ConcurrencyandThreads.Chapter4motivatesandexplainstheconceptof
threads.Becauseoftheincreasingimportanceofconcurrentprogramming,andits
integrationwithmodernprogramminglanguageslikeJava,manystudentshavebeen
introducedtomulti-threadedprogramminginanearlierclass.Thisisabitdangerous,
asstudentsatthisstagearepronetowritingprogramswithraceconditions,problems
thatmayormaynotbediscoveredwithtesting.Thus,thegoalofthischapteristo

provideasolidconceptualframeworkforunderstandingthesemanticsof
concurrency,aswellashowconcurrentthreadsareimplementedinboththe
operatingsystemkernelandinuser-levellibraries.Instructorsneedingtogomore
quicklycanomittheseimplementationdetails.
Synchronization.Chapter5discussesthesynchronizationofmulti-threaded
programs,acentralpartofalloperatingsystemsandincreasinglyimportantinmany
othercontexts.Ourapproachistodescribeoneeffectivemethodforstructuring
concurrentprograms(basedonMesamonitors),ratherthantoattempttocover
severaldifferentapproaches.Inourview,itismoreimportantforstudentstomaster
onemethodology.Monitorsareaparticularlyrobustandsimpleone,capableof
implementingmostconcurrentprogramsefficiently.Theimplementationof
synchronizationprimitivesshouldbeincludedifthereistime,sostudentsseethat
thereisnomagic.
Multi-ObjectSynchronization.Chapter6discussesadvancedtopicsinconcurrency
—specifically,thetwinchallengesofmultiprocessorlockcontentionanddeadlock.
Thismaterialisincreasinglyimportantforstudentsworkingonmulticoresystems,
butsomecoursesmaynothavetimetocoveritindetail.
Scheduling.Thischaptercoverstheconceptsofresourceallocationinthespecific
contextofprocessorscheduling.Withtheadventofdatacentercomputingand
multicorearchitectures,theprinciplesandpracticeofresourceallocationhave
renewedimportance.Afteraquicktourthroughthetradeoffsbetweenresponsetime
andthroughputforuniprocessorscheduling,thechaptercoversasetofmore


advancedtopicsinaffinityandmultiprocessorscheduling,power-awareanddeadline
scheduling,aswellasbasicqueueingtheoryandoverloadmanagement.Weconclude
thesetopicsbywalkingstudentsthroughacasestudyofserver-sideload
management.
AddressTranslation.Chapter8explainsmechanismsforhardwareandsoftware
addresstranslation.Thefirstpartofthechaptercovershowhardwareandoperating

systemscooperatetoprovideflexible,sparseaddressspacesthroughmulti-level
segmentationandpaging.Wethendescribehowtomakememorymanagement
efficientwithtranslationlookasidebuffers(TLBs)andvirtuallyaddressedcaches.
WeconsiderhowtokeepTLBsconsistentwhentheoperatingsystemmakeschanges
toitspagetables.Weconcludewithadiscussionofmodernsoftware-based
protectionmechanismssuchasthosefoundintheMicrosoftCommonLanguage
RuntimeandGoogle’sNativeClient.
CachingandVirtualMemory.Cachesarecentraltomanydifferenttypesof
computersystems.Moststudentswillhaveseentheconceptofacacheinanearlier
classonmachinestructures.Thus,ourgoalistocoverthetheoryandimplementation
ofcaches:whentheyworkandwhentheydonot,aswellashowtheyare
implementedinhardwareandsoftware.Wethenshowhowtheseideasareappliedin
thecontextofmemory-mappedfilesanddemand-pagedvirtualmemory.
AdvancedMemoryManagement.Addresstranslationisapowerfultoolinsystem
design,andweshowhowitcanbeusedforzerocopyI/O,virtualmachines,process
checkpointing,andrecoverablevirtualmemory.Asthisismoreadvancedmaterial,it
canbeskippedbythoseclassespressedfortime.
FileSystems:IntroductionandOverview.Chapter11framesthefilesystem
portionofthebook,startingtopdownwiththechallengesofprovidingausefulfile
abstractiontousers.WethendiscusstheUNIXfilesysteminterface,themajor
internalelementsinsideafilesystem,andhowdiskdevicedriversarestructured.
StorageDevices.Chapter12surveysblockstoragehardware,specificallymagnetic
disksandflashmemory.Thelasttwodecadeshaveseenrapidchangeinstorage
technologyaffectingbothapplicationprogrammersandoperatingsystemsdesigners;
thischapterprovidesasnapshotforstudents,asabuildingblockforthenexttwo
chapters.Ifstudentshavepreviouslyseenthismaterial,thischaptercanbeskipped.
FilesandDirectories.Chapter13discussesfilesystemlayoutondisk.Ratherthan
surveyallpossiblefilelayouts—somethingthatchangesrapidlyovertime—we
usefilesystemsasaconcreteexampleofmappingcomplexdatastructuresonto
blockstoragedevices.

ReliableStorage.Chapter14explainstheconceptandimplementationofreliable
storage,usingfilesystemsasaconcreteexample.Startingwiththeadhoctechniques
usedinearlyfilesystems,thechapterexplainscheckpointingandwriteahead
loggingasalternateimplementationstrategiesforbuildingreliablestorage,andit
discusseshowredundancysuchaschecksumsandreplicationareusedtoimprove
reliabilityandavailability.


Wewelcomeandencouragesuggestionsforhowtoimprovethepresentationofthe
material;pleasesendanycommentstothepublisher’swebsite,


Acknowledgements
Wehavebeenincrediblyfortunatetohavethehelpofalargenumberofpeopleinthe
conception,writing,editing,andproductionofthisbook.
WestartedonthejourneyofwritingthisbookoverdinnerattheUSENIXNSDI
conferencein2010.Atthetime,wethoughtperhapsitwouldtakeusthesummerto
completethefirstversionandperhapsayearbeforewecoulddeclareourselvesdone.We
wereverywrong!Itisnoexaggerationtosaythatitwouldhavetakenusalotlonger
withoutthehelpwehavereceivedfromthepeoplewementionbelow.
Perhapsmostimportanthavebeenourearlyadopters,whohavegivenusenormously
usefulfeedbackaswehaveputtogetherthisedition:
Carnegie-Mellon

DavidEckhardtandGarthGibson

Clarkson

JeannaMatthews

Cornell

GunSirer

ETHZurich

MothyRoscoe

NewYorkUniversity

LaskshmiSubramanian

PrincetonUniversity

KaiLi

SaarlandUniversity

PeterDruschel

StanfordUniversity

JohnOusterhout

UniversityofCaliforniaRiverside

HarshaMadhyastha

UniversityofCaliforniaSantaBarbara BenZhao
UniversityofMaryland

NeilSpring

UniversityofMichigan

PeteChen

UniversityofSouthernCalifornia

RameshGovindan

UniversityofTexas-Austin

LorenzoAlvisi


UniverstiyofToronto

DingYuan

UniversityofWashington

GaryKimuraandEdLazowska

Indevelopingourapproachtoteachingoperatingsystems,bothbeforewestartedwriting
andafterwardsaswetriedtoputourthoughtstopaper,wemadeextensiveuseoflecture
notesandslidesdevelopedbyotherfaculty.Ofparticularhelpwerethematerialscreated
byPeteChen,PeterDruschel,SteveGribble,EddieKohler,JohnOusterhout,Mothy
Roscoe,andGeoffVoelker.Wethankthemall.
Ourillustratorforthesecondedition,CameronNeat,hasbeenajoytoworkwith.

WearealsogratefultoLorenzoAlvisi,AdamAnderson,PeteChen,SteveGribble,Sam
Hopkins,EdLazowska,HarshaMadhyastha,JohnOusterhout,MarkRich,MothyRoscoe,
WillScott,GunSirer,IonStoica,LakshmiSubramanian,andJohnZahorjanfortheir
helpfulcommentsandsuggestionsastohowtoimprovethebook.
WethankJoshBerlin,MarlaDahlin,SandyKaplan,JohnOusterhout,WhitneySchmidt,
andMikeWalfishforhelpingusidentifyandcorrectgrammaticalortechnicalbugsinthe
text.
WethankJeffDean,GarthGibson,MarkOskin,SimonPeter,DaveProbert,AminVahdat,
andMarkZbikowskifortheirhelpinexplainingtheinternalworkingsofsomeofthe
commercialsystemsmentionedinthisbook.
WewouldliketothankDaveWetherall,DanWeld,MikeWalfish,DavePatterson,Olav
Kvern,DanHalperin,ArmandoFox,RobinBriggs,KatyaAnderson,SandraAnderson,
LorenzoAlvisi,andWilliamAdamsfortheirhelpandadviceontextbookeconomicsand
production.
TheHelenRiaboffWhiteleyCenteraswellasDonandJeanneDahlinwerekindenough
tolendusaplacetoescapewhenweneededtogetchapterswritten.
Finally,wethankourfamilies,ourcolleagues,andourstudentsforsupportingusinthis
larger-than-expectedeffort.



I
KernelsandProcesses



1.Introduction
AllIreallyneedtoknowIlearnedinkindergarten.—RobertFulgham

Howdoweconstructreliable,portable,efficient,andsecurecomputersystems?An

essentialcomponentisthecomputer’soperatingsystem—thesoftwarethatmanagesa
computer’sresources.
First,thebadnews:operatingsystemsconceptsareamongthemostcomplexincomputer
science.Amodern,general-purposeoperatingsystemcanexceed50millionlinesofcode,
orinotherwords,morethanathousandtimeslongerthanthistextbook.Newoperating
systemsarebeingwrittenallthetime:ifyouuseane-bookreader,tablet,orsmartphone,
anoperatingsystemismanagingyourdevice.Giventhisinherentcomplexity,welimitour
focustotheessentialconceptsthateverycomputerscientistshouldknow.
Nowthegoodnews:operatingsystemsconceptsarealsoamongthemostaccessiblein
computerscience.Manytopicsinthisbookwillseemfamiliartoyou—ifyouhaveever
triedtodotwothingsatonce,orpickedthe“wrong”lineatagrocerystore,ortriedto
keeparoommateorsiblingfrommessingwithyourthings,orsucceededatpullingoffan
AprilFool’sjoke.Eachoftheseactivitieshasananalogueinoperatingsystems.Itisthis
familiaritythatgivesushopethatwecanexplainhowoperatingsystemsworkinasingle
textbook.Allweassumeofthereaderisabasicunderstandingoftheoperationofa
computerandtheabilitytoreadpseudo-code.
Webelievethatunderstandinghowoperatingsystemsworkisessentialforanystudent
interestedinbuildingmoderncomputersystems.Ofcourse,everyonewhousesa
computerorasmartphone—orevenamoderntoaster—usesanoperatingsystem,so
understandingthefunctionofanoperatingsystemisusefultomostcomputerscientists.
Thisbookaimstogomuchdeeperthanthat,toexplainoperatingsysteminternalsthatwe
relyoneverydaywithoutrealizingit.
Softwareengineersusemanyofthesametechnologiesanddesignpatternsasthoseused
inoperatingsystemstobuildothercomplexsystems.Whetheryourgoalistoworkonthe
internalsofanoperatingsystemkernel—ortobuildthenextgenerationofsoftwarefor
cloudcomputing,securewebbrowsers,gameconsoles,graphicaluserinterfaces,media
players,databases,ormulticoresoftware—theconceptsandabstractionsneededfor
reliable,portable,efficientandsecuresoftwarearemuchthesame.Inourexperience,the
bestwaytolearntheseconceptsistostudyhowtheyareusedinoperatingsystems,but
wehopeyouwillapplythemtoamuchbroaderrangeofcomputersystems.

Togetstarted,considerthewebserverinFigure1.1.Itsbehaviorisamazinglysimple:it
receivesapacketcontainingthenameofthewebpagefromthenetwork,asanHTTP
GETrequest.Thewebserverdecodesthepacket,readsthefilefromdisk,andsendsthe
contentsofthefilebackoverthenetworktotheuser’smachine.


Figure1.1:Theoperationofawebserver.TheclientmachinesendsanHTTPGETrequesttothewebserver.The
serverdecodesthepacket,readsthefile,andsendsthecontentsbacktotheclient.

Partofanoperatingsystem’sjobistomakeiteasytowriteapplicationslikewebservers.
Butdiggingabitdeeper,thissimplestoryquicklyraisesasmanyquestionsasitanswers:
Manywebrequestsinvolvebothdataandcomputation.Forexample,theGoogle
homepagepresentsasimpletextbox,buteachsearchqueryenteredinthatbox
consultsdataspreadovermanymachines.Tokeeptheirsoftwaremanageable,web
serversofteninvokehelperapplications,e.g.,tomanagetheactualsearchfunction.
Themainwebservermustbeabletocommunicatewiththehelperapplicationsfor
thistowork.Howdoestheoperatingsystemenablemultipleapplicationsto
communicatewitheachother?
Whatiftwousers(oramillion)requestawebpagefromtheserveratthesametime?
Asimpleapproachmightbetohandleeachrequestinturn.Ifanyindividualrequest
takesalongtime,however,everyotherrequestmustwaitforittocomplete.Afaster,
butmorecomplex,solutionistomultitask:tojugglethehandlingofmultiplerequests
atonce.Multitaskingisespeciallyimportantonmodernmulticorecomputers,where
eachprocessorcanhandleadifferentrequestatthesametime.Howdoesthe
operatingsystemenableapplicationstodomultiplethingsatonce?
Forbetterperformance,thewebservermightwanttokeepacopy,sometimescalled
acache,ofrecentlyrequestedpages.Inthisway,ifmultipleusersrequestthesame
page,theservercanrespondtosubsequentrequestsmorequicklyfromthecache,
ratherthanstartingeachrequestfromscratch.Thisrequiresthewebserverto
coordinate,orsynchronize,accesstothecache’sdatastructuresbypossibly

thousandsofwebrequestsatthesametime.Howdoestheoperatingsystem
synchronizeapplicationaccesstoshareddata?
Tocustomizeandanimatetheuserexperience,webserverstypicallysendclients
scriptingcodealongwiththecontentsofthewebpage.Butthismeansthatclicking
onalinkcancausesomeoneelse’scodetorunonyourcomputer.Howdoesthe
clientoperatingsystemprotectitselffromcompromisebyacomputervirus


surreptitiouslyembeddedintothescriptingcode?
Supposethewebsiteadministratorusesaneditortoupdatethewebpage.Theweb
servermustbeabletoreadthisfile.Howdoestheoperatingsystemstorethebyteson
disksothatthewebservercanfindandreadthem?
Takingthisastepfurther,theadministratormaywanttomakeaconsistentsetof
changestothewebsitesothatembeddedlinksarenotleftdangling,even
temporarily.Howcantheoperatingsystemletusersmakeasetofchangestoaweb
site,sothatrequestsseeeithertheoldornewpages,butnotacombinationofthe
two?
Whathappenswhentheclientbrowserandthewebserverrunatdifferentspeeds?If
theservertriestosendawebpagetotheclientfasterthantheclientcanrenderthe
pageonthescreen,wherearethecontentsofthefilestoredinthemeantime?Canthe
operatingsystemdecoupletheclientandserversothateachcanrunatitsownspeed
withoutslowingtheotherdown?
Asdemandonthewebservergrows,theadministratormayneedtomovetomore
powerfulhardware,withmorememory,moreprocessors,fasternetworkdevices,and
fasterdisks.Totakeadvantageofnewhardware,mustthewebserverbere-written
eachtime,orcanitbewritteninahardware-independentfashion?Whataboutthe
operatingsystem—mustitbere-writtenforeverynewpieceofhardware?
Wecouldgoon,butyougettheidea.Thisbookwillhelpyouunderstandtheanswersto
theseandmanymorequestions.
Chapterroadmap:

Therestofthischapterdiscussesthreetopicsindetail:
OperatingSystemDefinition.Whatisanoperatingsystem,andwhatdoesitdo?
(Section1.1)
OperatingSystemEvaluation.Whatdesigngoalsshouldwelookforinan
operatingsystem?(Section1.2)
OperatingSystems:Past,Present,andFuture.Howhaveoperatingsystems
evolved,andwhatnewfunctionalityarewelikelytoseeinfutureoperatingsystems?
(Section1.3)

1.1WhatIsAnOperatingSystem?
Anoperatingsystem(OS)isthelayerofsoftwarethatmanagesacomputer’sresourcesfor
itsusersandtheirapplications.Operatingsystemsruninawiderangeofcomputer
systems.Theymaybeinvisibletotheenduser,controllingembeddeddevicessuchas
toasters,gamingsystems,andthemanycomputersinsidemodernautomobilesand
airplanes.Theyarealsoessentialtomoregeneral-purposesystemssuchassmartphones,
desktopcomputers,andservers.
Ourdiscussionwillfocusongeneral-purposeoperatingsystemsbecausethetechnologies


theyneedareasupersetofthoseneededforembeddedsystems.Increasingly,operating
systemstechnologiesdevelopedforgeneral-purposecomputingaremigratingintothe
embeddedsphere.Forexample,earlymobilephoneshadsimpleoperatingsystemsto
managetheirhardwareandtorunahandfulofprimitiveapplications.Today,smartphones
—phonescapableofrunningindependentthird-partyapplications—arethefastest
growingsegmentofthemobilephonebusiness.Thesedevicesrequiremuchmore
completeoperatingsystems,withsophisticatedresourcemanagement,multi-tasking,
securityandfailureisolation.
Likewise,automobilesareincreasinglysoftwarecontrolled,raisingahostofoperating
systemissues.Cananyonewritesoftwareforyourcar?Whatifthesoftwarefailswhile
youaredrivingdownthehighway?Canacar’soperatingsystembehijackedbya

computervirus?Althoughthismightseemfar-fetched,researchersrecentlydemonstrated
thattheycouldremotelyturnoffacar’sbrakingsystemthroughacomputervirus
introducedintothecar’scomputersviaahackedcarradio.Agoalofthisbookisto
explainhowtobuildmorereliableandsecurecomputersystemsinavarietyofcontexts.

Figure1.2:Ageneral-purposeoperatingsystemisalayerofsoftwarethatmanagesacomputer’sresourcesforitsusers
andapplications.

Forgeneral-purposesystems,usersinteractwithapplications,applicationsexecuteinan
environmentprovidedbytheoperatingsystem,andtheoperatingsystemmediatesaccess
totheunderlyinghardware,asshowninFigure1.2andexpandedinFigure1.3.Howcan
anoperatingsystemrunmultipleapplications?Forthis,operatingsystemsneedtoplay
threeroles:


Figure1.3:Thisshowsthestructureofageneral-purposeoperatingsystem,asanexpansiononthesimpleview
presentedinFigure1.2.Atthelowestlevel,thehardwareprovidesprocessors,memory,andasetofdevicesforstoring
dataandcommunicatingwiththeoutsideworld.Thehardwarealsoprovidesprimitivesthattheoperatingsystemcan
useforfaultisolationandsynchronization.Theoperatingsystemrunsasthelowestlayerofsoftwareonthecomputer.
Itcontainsbothadevice-specificlayerformanagingthemyriadhardwaredevicesandasetofdevice-independent
servicesprovidedtoapplications.Sincetheoperatingsystemmustisolatemaliciousandbuggyapplicationsfromother
applicationsortheoperatingsystemitself,muchoftheoperatingsystemrunsinaseparateexecutionenvironment
protectedfromapplicationcode.Aportionoftheoperatingsystemcanalsorunasasystemlibrarylinkedintoeach
application.Inturn,applicationsruninanexecutioncontextprovidedbytheoperatingsystemkernel.Theapplication
contextismuchmorethanasimpleabstractionontopofhardwaredevices:applicationsexecuteinavirtual
environmentthatismoreconstrained(topreventharm),morepowerful(tomaskhardwarelimitations),andmoreuseful
(viacommonservices)thantheunderlyinghardware.

1. Referee.Operatingsystemsmanageresourcessharedbetweendifferentapplications
runningonthesamephysicalmachine.Forexample,anoperatingsystemcanstop