Deep learning with theano

DeepLearningwithTheano


TableofContents
DeepLearningwithTheano
Credits
AbouttheAuthor
Acknowledgments
AbouttheReviewers
www.PacktPub.com
eBooks,discountoffers,andmore
Whysubscribe?
CustomerFeedback
Preface
Whatthisbookcovers
WhyTheano?
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
Customersupport
Downloadingtheexamplecode
Errata
Piracy
Questions
1.TheanoBasics
Theneedfortensors
InstallingandloadingTheano
Condapackageandenvironmentmanager
InstallingandrunningTheanoonCPU

GPUdriversandlibraries
InstallingandrunningTheanoonGPU
Tensors
Graphsandsymboliccomputing
Operationsontensors
Dimensionmanipulationoperators
Elementwiseoperators
Reductionoperators
Linearalgebraoperators
Memoryandvariables
Functionsandautomaticdifferentiation
Loopsinsymboliccomputing
Configuration,profilinganddebugging
Summary
2.ClassifyingHandwrittenDigitswithaFeedforwardNetwork


TheMNISTdataset
Structureofatrainingprogram
Classificationlossfunction
Single-layerlinearmodel
Costfunctionanderrors
Backpropagationandstochasticgradientdescent
Multiplelayermodel
Convolutionsandmaxlayers
Training
Dropout
Inference
Optimizationandotherupdaterules
Relatedarticles

Summary
3.EncodingWordintoVector
Encodingandembedding
Dataset
ContinuousBagofWordsmodel
Trainingthemodel
Visualizingthelearnedembeddings
Evaluatingembeddings–analogicalreasoning
Evaluatingembeddings–quantitativeanalysis
Applicationofwordembeddings
Weighttying
Furtherreading
Summary
4.GeneratingTextwithaRecurrentNeuralNet
NeedforRNN
Adatasetfornaturallanguage
Simplerecurrentnetwork
LSTMnetwork
Gatedrecurrentnetwork
Metricsfornaturallanguageperformance
Traininglosscomparison
Exampleofpredictions
ApplicationsofRNN
Relatedarticles
Summary
5.AnalyzingSentimentwithaBidirectionalLSTM
InstallingandconfiguringKeras
ProgrammingwithKeras
SemEval2013dataset
Preprocessingtextdata

Designingthearchitectureforthemodel
Vectorrepresentationsofwords


Sentencerepresentationusingbi-LSTM
Outputtingprobabilitieswiththesoftmaxclassifier
Compilingandtrainingthemodel
Evaluatingthemodel
Savingandloadingthemodel
Runningtheexample
Furtherreading
Summary
6.LocatingwithSpatialTransformerNetworks
MNISTCNNmodelwithLasagne
Alocalizationnetwork
Recurrentneuralnetappliedtoimages
Unsupervisedlearningwithco-localization
Region-basedlocalizationnetworks
Furtherreading
Summary
7.ClassifyingImageswithResidualNetworks
Naturalimagedatasets
Batchnormalization
Globalaveragepooling
Residualconnections
Stochasticdepth
Denseconnections
Multi-GPU
Dataaugmentation
Furtherreading

Summary
8.TranslatingandExplainingwithEncoding–decodingNetworks
Sequence-to-sequencenetworksfornaturallanguageprocessing
Seq2seqfortranslation
Seq2seqforchatbots
Improvingefficiencyofsequence-to-sequencenetwork
Deconvolutionsforimages
Multimodaldeeplearning
Furtherreading
Summary
9.SelectingRelevantInputsorMemorieswiththeMechanismofAttention
Differentiablemechanismofattention
Bettertranslationswithattentionmechanism
Betterannotateimageswithattentionmechanism
StoreandretrieveinformationinNeuralTuringMachines
Memorynetworks
Episodicmemorywithdynamicmemorynetworks
Furtherreading
Summary


10.PredictingTimesSequenceswithAdvancedRNN
DropoutforRNN
DeepapproachesforRNN
Stackedrecurrentnetworks
Deeptransitionrecurrentnetwork
Highwaynetworksdesignprinciple
RecurrentHighwayNetworks
Furtherreading
Summary

11.LearningfromtheEnvironmentwithReinforcement
Reinforcementlearningtasks
Simulationenvironments
Q-learning
DeepQ-network
Trainingstability
PolicygradientswithREINFORCEalgorithms
Relatedarticles
Summary
12.LearningFeatureswithUnsupervisedGenerativeNetworks
Generativemodels
RestrictedBoltzmannMachines
Deepbeliefbets
Generativeadversarialnetworks
ImproveGANs
Semi-supervisedlearning
Furtherreading
Summary
13.ExtendingDeepLearningwithTheano
TheanoOpinPythonforCPU
TheanoOpinPythonfortheGPU
TheanoOpinCforCPU
TheanoOpinCforGPU
Coalescedtransposeviasharedmemory,NVIDIAparallelforall
Modelconversions
Thefutureofartificialintelligence
Furtherreading
Summary
Index

DeepLearningwithTheano


DeepLearningwithTheano
Copyright©2017PacktPublishingAllrightsreserved.Nopartofthisbookmaybereproduced,storedin
aretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthe
publisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformation
presented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressor
implied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliablefor
anydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesand
productsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannot
guaranteetheaccuracyofthisinformation.
Firstpublished:July2017
Productionreference:1280717
PublishedbyPacktPublishingLtd.
LiveryPlace
35LiveryStreet
BirminghamB32PB,UK.
ISBN978-1-78646-582-5
www.packtpub.com


Credits
Author
ChristopherBourez
Reviewers
MatthieudeBeaucorps

FredericBastien
ArnaudBergeron
PascalLamblin
CommissioningEditor
AmeyVarangaonkar
AcquisitionEditor
VeenaPagare
ContentDevelopmentEditor
AmritaNoronha
TechnicalEditor
AkashPatel
CopyEditor
SafisEditing
ProjectCoordinator
ShwetaHBirwatkar
Proofreader
SafisEditing
Indexer
PratikShirodkar


Graphics
TaniaDutta
ProductionCoordinator
ShantanuZagade
CoverWork
ShantanuN.Zagade


AbouttheAuthor

ChristopherBourezgraduatedfromEcolePolytechniqueandEcoleNormaleSupérieuredeCachanin
Parisin2005withaMasterofScienceinMath,MachineLearningandComputerVision(MVA).
For7years,heledacompanyincomputervisionthatlaunchedPixee,avisualrecognitionapplicationfor
iPhonein2007,withthemajormovietheaterbrand,thecityofParisandthemajorticketbroker:witha
snapofapicture,theusercouldgetinformationaboutevents,products,andaccesstopurchase.
WhileworkingonmissionsincomputervisionwithCaffe,TensorFloworTorch,hehelpedother
developerssucceedbywritingonablogoncomputerscience.Oneofhisblogposts,atutorialonthe
Caffedeeplearningtechnology,hasbecomethemostsuccessfultutorialonthewebaftertheofficialCaffe
website.
OntheinitiativeofPacktPublishing,thesamerecipesthatmadethesucessofhisCaffetutorialhavebeen
portedtowritethisbookonTheanotechnology.Inthemeantime,awiderangeofproblemsforDeep
LearningarestudiedtogainmorepracticewithTheanoanditsapplication.


Acknowledgments
Thisbookhasbeenwritteninlessthanayear,andIwouldliketothankMohammedJabreelforhishelp
withwritingtextsandcodeexamplesforchapters3and5.
MohammedHamoodJabreelisisaPhDstudentinComputerScienceEngineeringattheDepartmentof
ComputerScienceandMathematics,UniversitatRoviraiVirgili.HehasreceivedaMasterdegreein
ComputerEngineering:ComputerSecurityandIntelligentSystemsfromUniversitatRoviraiVirgili,
Spainin2015andaBachelor'sdegreeinComputerSciencein2009fromHodiedhaUniversity.Hismain
researchinterestistheNaturalLanguageProcessing,TextMiningandSentimentAnalysis.
Second,IwouldliketothankIBMfortheirtremendoussupportthroughtheGlobalEntrepeneurProgram.
TheirinfrastructureofdedicatedGPUshasbeenofuncomparablequalityandperformancetotrainthe
neuralnetworks.
Last,Iwouldliketothankthereviewers,MatthieudeBeaucorpsandPascalLamblin,aswellasthePackt
employeesAmritaandVinayfortheirideasandfollow-up.
Happyreading.

AbouttheReviewers
MatthieudeBeaucorpsisamachinelearningspecialistwithanengineeringbackground.Since2012,he
hasbeenworkingondevelopingdeepneuralnetstoenhanceidentificationandrecommendationtasksin
computervision,audio,andNLP.
PascalLamblinisasoftwareanalystatMILA(MontrealInstituteforLearningAlgorithms).After
completinghisengineeringdegreeatÉcoleCentraleParis,Pascalhasdonesomeresearchunderthe
supervisionofYoshuaBengioatUniversitédeMontréalandisnowworkingonthedevelopmentof
Theano.


www.PacktPub.com
eBooks,discountoffers,andmore
Forsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfiles
available?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,
youareentitledtoadiscountontheeBookcopy.Getintouchwithusat
<>formoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeof
freenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.


/>Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksand
videocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvance
yourcareer.


Whysubscribe?
FullysearchableacrosseverybookpublishedbyPackt
Copyandpaste,print,andbookmarkcontent
Ondemandandaccessibleviaawebbrowser

CustomerFeedback
ThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpus
improve,pleaseleaveusanhonestreviewonthisbook'sAmazonpageat
Ifyou'dliketojoinourteamofregularreviewers,youcanemailusatWeawardourregularreviewerswithfreeeBooksand
videosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!


Preface
Gaininsightandpracticewithneuralnetarchitecturedesigntosolveproblemswithartificialintelligence.
Understandtheconceptsbehindthemostadvancednetworksindeeplearning.LeveragePythonlanguage
withTheanotechnology,toeasilycomputederivativesandminimizeobjectivefunctionsofyourchoice.


Whatthisbookcovers
Chapter1,TheanoBasics,helpsthereadertoreaderlearnmainconceptsofTheanotowritecodethat
cancompileondifferenthardwarearchitecturesandoptimizeautomaticallycomplexmathematical
objectivefunctions.
Chapter2,ClassifyingHandwrittenDigitswithaFeedforwardNetwork,willintroduceasimple,wellknownandhistoricalexamplewhichhasbeenthestartingproofofsuperiorityofdeeplearning
algorithms.Theinitialproblemwastorecognizehandwrittendigits.
Chapter3,EncodingwordintoVector,oneofthemainchallengewithneuralnetsistoconnectthereal
worlddatatotheinputofaneuralnet,inparticularforcategoricalanddiscretedata.Thischapter
presentsanexampleonhowtobuildanembeddingspacethroughtrainingwithTheano.
Suchembeddingsareveryusefulinmachinetranslation,robotics,imagecaptioning,andsoonbecause
theytranslatetherealworlddataintoarraysofvectorsthatcanbeprocessedbyneuralnets.
Chapter4,GeneratingTextwithaRecurrentNeuralNet,introducesrecurrencyinneuralnetswitha
simpleexampleinpractice,togeneratetext.
Recurrentneuralnets(RNN)areapopulartopicindeeplearning,enablingmorepossibilitiesfor
sequenceprediction,sequencegeneration,machinetranslation,connectedobjects.NaturalLanguage

Processing(NLP)isasecondfieldofinterestthathasdriventheresearchfornewmachinelearning
techniques.
Chapter5,AnalyzingSentimentswithaBidirectionalLSTM,appliesembeddingsandrecurrentlayersto
anewtaskofnaturallanguageprocessing,sentimentanalysis.Itactsasakindofvalidationofprior
chapters.
Inthemeantime,itdemonstratesanalternativewaytobuildneuralnetsonTheano,withahigherlevel
library,Keras.
Chapter6,LocatingwithSpatialTransformerNetworks,appliesrecurrencytoimage,toreadmultiple
digitsonapageatonce.Thistime,wetaketheopportunitytorewritetheclassificationnetworkfor
handwrittendigitsimages,andourrecurrentmodels,withthehelpofLasagne,alibraryofbuilt-in
modulesfordeeplearningwithTheano.
Lasagnelibraryhelpsdesignneuralnetworksforexperimentingfaster.Withthishelp,we'lladdress
objectlocalization,acommoncomputervisionchallenge,withSpatialTransformermodulestoimprove
ourclassificationscores.
Chapter7,ClassifyingImageswithResidualNetworks,classifiesanytypeofimagesatthebest
accuracy.Inthemeantime,tobuildmorecomplexnetswithease,weintroducealibrarybasedonTheano
framework,Lasagne,withmanyalreadyimplementedcomponentstohelpimplementneuralnetsfasterfor
Theano.


Chapter8,TranslatingandExplainingthroughEncoding–decodingNetworks,presentsencodingdecodingtechniques:appliedtotext,thesetechniquesareheavilyusedinmachine-translationandsimple
chatbotssystems.Appliedtoimages,theyservescenesegmentationsandobjectlocalization.Last,image
captioningisamixed,encodingimagesanddecodingtotexts.
Thischaptergoesonestepfurtherwithaverypopularhighlevellibrary,Keras,thatsimplifiesevenmore
thedevelopmentofneuralnetswithTheano.
Chapter9,SelectingRelevantInputsorMemorieswiththeMechanismofAttention,forsolvingmore
complicatedtasks,themachinelearningworldhasbeenlookingforhigherlevelofintelligence,inspired
bynature:reasoning,attentionandmemory.Inthischapter,thereaderwilldiscoverthememorynetworks
onthemainpurposeofartificialintelligencefornaturallanguageprocessing(NLP):thelanguage
understanding.

Chapter10,PredictingTimesSequencewithAdvancedRNN,timesequencesareanimportantfield
wheremachinelearninghasbeenusedheavily.Thischapterwillgoforadvancedtechniqueswith
RecurrentNeuralNetworks(RNN),togetstate-of-artresults.
Chapter11,LearningfromtheEnvironmentwithReinforcement,reinforcementlearningisthevastarea
ofmachinelearning,whichconsistsintraininganagenttobehaveinanenvironment(suchasavideo
game)soastooptimizeaquantity(maximizingthegamescore),byperformingcertainactionsinthe
environment(pressingbuttonsonthecontroller)andobservingwhathappens.
Reinforcementlearningnewparadigmopensacompletenewpathfordesigningalgorithmsand
interactionsbetweencomputersandrealworld.
Chapter12,LearningFeatureswithUnsupervisedGenerativeNetworks,unsupervisedlearningconsists
innewtrainingalgorithmsthatdonotrequirethedatatobelabeledtobetrained.Thesealgorithmstryto
inferthehiddenlabelsfromthedata,calledthefactors,and,forsomeofthem,togeneratenewsynthetic
data.
Unsupervisedtrainingisveryusefulinmanycases,eitherwhennolabelingexists,orwhenlabelingthe
datawithhumansistooexpensive,orlastlywhenthedatasetistoosmallandfeatureengineeringwould
overfitthedata.Inthislastcase,extraamountsofunlabeleddatatrainbetterfeaturesasabasisfor
supervisedlearning.
Chapter13,ExtendingDeepLearningwithTheano,extendsthesetofpossibilitiesinDeepLearning
withTheano.Itaddressesthewaytocreatenewoperatorsforthecomputationgraph,eitherinPythonfor
simplicity,orinCtoovercomethePythonoverhead,eitherfortheCPUorfortheGPU.Also,introduces
thebasicconceptofparallelprogrammingforGPU.Lastly,weopenthefieldofGeneralIntelligence,
basedonthefirstskillsdeveloppedinthisbook,todevelopnewskills,inagradualway,toimprove
itselfonestepfurther.


WhyTheano?
InvestingtimeanddevelopmentsonTheanoisveryvaluableandtounderstandwhy,itisimportantto
explainthatTheanobelongstothebestdeeplearningtechnologiesandisalsomuchmorethanadeep
learninglibrary.ThreereasonsmakeofTheanoagoodchoiceofinvestment:
Ithascomparableperformancewithothernumericalordeeplearninglibraries

ItcomesinarichPythonecosystem
Itenablesyoutoevaluateanyfunctionconstraintbydata,givenamodel,byleavingthefreedomto
compileasolutionforanyoptimizationproblem
Letusfirstfocusontheperformanceofthetechnologyitself.Themostpopularlibrariesindeeplearning
areTheano(forPython),Torch(forLua),Tensorflow(forPython)andCaffe(forC++andwithaPython
wrapper).Therehasbeenlot'sofbenchmarkstocomparedeeplearningtechnologies.
InBastienetal2012(Theano:newfeaturesandspeedimprovements,FrédéricBastien,Pascal
Lamblin,RazvanPascanu,JamesBergstra,IanGoodfellow,ArnaudBergeron,NicolasBouchard,
DavidWarde-Farley,YoshuaBengio,Nov2012),Theanomadesignificantprogressinspeed,butthe
comparisonondifferenttasksdoesnotpointaclearwinneramongthechallengedtechnologies.
BahrampouretAl.2016(ComparativeStudyofDeepLearningSoftwareFrameworks,Soheil
Bahrampour,NaveenRamakrishnan,LukasSchott,MohakShah,mars2016)concludethat:
ForGPU-baseddeploymentoftrainedconvolutionalandfullyconnectednetworks,Torchisbest
suited,followedbyTheano.
ForGPU-basedtrainingofconvolutionalandfullyconnectednetworks,Theanoisfastestforsmall
networksandTorchisfastestforlargernetworks
ForGPU-basedtraininganddeploymentofrecurrentnetworks(LSTM),Theanoresultsinthebest
performance.
ForCPU-basedtraininganddeploymentofanytesteddeepnetworkarchitecture,Torchperformsthe
bestfollowedbyTheano
Theseresultsareconfirmedintheopen-sourcernn-benchmarks(wherefortraining(forward+backwardpasses),TheanooutperformsTorchand
TensorFlow.Also,TheanocrushesTorchandTensorFlowforsmallerbatchsizeswithlargernumbersof
hiddenunits.Forbiggerbatchsizeandhiddenlayersize,thedifferencesaresmallersincetheyrelymore
ontheperformanceofCUDA,theunderlyingNVIDIAgraphiclibrarycommontoallframeworks.Last,in
up-to-datesoumithbenchmarks(thefftconvinTheano
performsthebestonCPU,whilethebestperformingconvolutionimplementationsonGPU,cudaconvnet2andfbfft,areCUDAextension,theunderlyinglibrary.Theseresultsshouldconvincethereader
that,althoughresultsaremixed,Theanoplaysaleadingroleinthespeedcompetition.
ThesecondpointtopreferTheanoratherthanTorchisthatitcomeswitharichecosystem,takingbenefit
fromthePythonecosystem,butalsofromalargenumberoflibrariesthathavebeendevelopedfor
Theano.Thisbookwillpresenttwoofthem,Lasagne,andKeras.TheanoandTorcharethemost

extensibleframeworksbothintermsofsupportingvariousdeeparchitecturesbutalsointermsof
supportedlibraries.Last,Theanohasnotareputationtobecomplextodebug,contrarytootherdeep


learninglibraries.
ThethirdpointmakesTheanoanuncomparabletoolforthecomputerscientistbecauseitisnotspecificto
deeplearning.AlthoughTheanopresentsthesamemethodsfordeeplearningthanotherlibraries,its
underlyingprinciplesareverydifferent:infact,Theanocompilesthecomputationgraphonthetarget
architecture.ThiscompilationstepmakesTheano'sspecificity,anditshouldbedefinedasa
mathematicalexpressioncompiler,designedwithmachinelearninginmind.Thesymbolic
differentiationisoneofthemostusefulfeaturesthatTheanooffersforimplementingnon-standarddeep
architectures.Therefore,Theanoisabletoaddressamuchlargerrangeofnumericalproblems,andcan
beusedtofindthesolutionthatminimizesanyproblemexpressedwithadifferentiablelossorenergy
function,givenanexistingdataset.


Whatyouneedforthisbook
Theanoinstallationrequirescondaorpip,andtheinstallprocessisthesameunderWindows,MacOS
andLinux.
ThecodehasbeentestedunderMacOSandLinuxUbuntu.Theremightbesomespecificitiesfor
Windows,suchasmodifyingthepaths,thattheWindowsdeveloperwillsolvequiteeasily.
Codeexamplessupposethereexistsonyourcomputerasharedfolder,wheretodownload,uncompress,
andpreprocessdatabasefilesthatcanbeveryvoluminousandshouldnotbeleftinsidecoderepositories.
Thispracticehelpssparesomediskspace,whilemultiplecodedirectoriesanduserscanusethesame
copyofthedatabase.Thefolderisusuallysharedbetweenuserspaces:
sudomkdir/sharedfiles
sudochmod777/sharedfiles


Whothisbookisfor

Thisbookisindentedtoprovidethewidestoverviewofdeeplearning,withTheanoassupport
technology.Thebookisdesignedforthebeginnerindeeplearningandartificialintelligence,aswellas
thecomputerprogrammerwhowantstogetacrossdomainexperienceandbecomefamiliarwithTheano
anditssupportinglibraries.Thisbookhelpsthereadertobeginwithdeeplearning,aswellasgettingthe
relevantandpracticalinformationsindeeplearning.
ArerequiredsomebasicskillsinPythonprogrammingandcomputerscience,aswellasskillsin
elementaryalgebraandcalculus.TheunderlyingtechnologyforallexperimentsisTheano,andthebook
providesfirstanin-depthpresentationofthecoretechnologyfirst,thenintroduceslateronsomelibraries
todosomereuseofexistingmodules.
Theapproachofthisbookistointroducethereadertodeeplearning,describingthedifferenttypesof
networksandtheirapplications,andinthemeantime,exploringthepossibilitiesofferedbyTheano,a
deeplearningtechnology,thatwillbethesupportforallimplementations.Thisbooksumsupsomeofthe
bestperformingnetsandstateoftheartresultsandhelpsthereadergettheglobalpictureofdeep
learning,takingherfromthesimpletothecomplexnetsgradually.
SincePythonhasbecomethemainprogramminglanguageindatascience,thisbooktriestocoverallthat
aPythonprogrammerneedstoknowtododeeplearningwithPythonandTheano.
ThebookwillintroducetwoabstractionframeworksontopofTheano,LasagneandKeras,whichcan
simplifythedevelopmentofmorecomplexnets,butdonotpreventyoufromunderstandingtheunderlying
concepts.


Conventions
Inthisbook,youwillfindanumberoftextstylesthatdistinguishbetweendifferentkindsofinformation.
Herearesomeexamplesofthesestylesandanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummy
URLs,userinput,andTwitterhandlesareshownasfollows:"Theoperatorisdefinedbyaclassderiving
fromthegenerictheano.Opclass."
Ablockofcodeissetasfollows:
importtheano,numpy
classAXPBOp(theano.Op):

"""
ThiscreatesanOpthattakesxtoa*x+b.
"""
__props__=("a","b")

Anycommand-lineinputoroutputiswrittenasfollows:
gsutilmb-leurope-west1gs://keras_sentiment_analysis

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,forexample,in
menusordialogboxes,appearinthetextlikethis:"ClickingtheNextbuttonmovesyoutothenext
screen."

Note
Warningsorimportantnotesappearinaboxlikethis.

Tip
Tipsandtricksappearlikethis.


Readerfeedback
Feedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyou
likedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallyget
themostoutof.
Tosendusgeneralfeedback,simplye-mail<>,andmentionthebook'stitlein
thesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoa
book,seeourauthorguideatwww.packtpub.com/authors.