PostgreSQL server programming second edition by usama dar

PostgreSQLServerProgrammingSecond
Edition


TableofContents
PostgreSQLServerProgrammingSecondEdition
Credits
AbouttheAuthors
AbouttheReviewers
www.PacktPub.com
Supportfiles,eBooks,discountoffers,andmore
Whysubscribe?
FreeaccessforPacktaccountholders
Preface
Whatthisbookcovers
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
Customersupport
Downloadingtheexamplecode
Errata
Piracy
Questions
1.WhatIsaPostgreSQLServer?
Whyprogramintheserver?
UsingPL/pgSQLforintegritychecks
Aboutthisbook’scodeexamples
Switchingtotheexpandeddisplay

Movingbeyondsimplefunctions
Datacomparisonsusingoperators
Managingrelateddatawithtriggers
Auditingchanges
Datacleaning


Customsortorders
Programmingbestpractices
KISS–keepitsimplestupid
DRY–don’trepeatyourself
YAGNI–youain’tgonnaneedit
SOA–service-orientedarchitecture
Typeextensibility
Caching
Wrappingup–whyprogramintheserver?
Performance
Easeofmaintenance
Improvedproductivity
Simplewaystotightensecurity
Summary
2.ServerProgrammingEnvironments
Costofacquisition
Availabilityofdevelopers
Licensing
Predictability
Community
Procedurallanguages
Third-partytools
Platformcompatibility

Applicationdesign
Databasesareconsideredharmful
Encapsulation
WhatdoesPostgreSQLoffer?
Datalocality
Morebasics
Transactions
Generalerrorreportinganderrorhandling


User-definedfunctions
Otherparameters
Morecontrol
Summary
3.YourFirstPL/pgSQLFunction
WhyPL/pgSQL?
ThestructureofaPL/pgSQLfunction
Accessingfunctionarguments
Conditionalexpressions
Loopswithcounters
Statementtermination
Loopingthroughqueryresults
PERFORMversusSELECT
LoopingThroughArrays
Returningarecord
Actingonthefunction’sresults
Summary
4.ReturningStructuredData
Setsandarrays
Returningsets

Returningasetofintegers
Usingasetreturningfunction
Functionsbasedonviews
OUTparametersandrecords
OUTparameters
Returningrecords
UsingRETURNSTABLE
Returningwithnopredefinedstructure
ReturningSETOFANY
Variadicargumentlists
AsummaryoftheRETURNSETOFvariants


Returningcursors
Iteratingovercursorsreturnedfromanotherfunction
Wrappingupoffunctionsreturningcursors
Otherwaystoworkwithstructureddata
Complexdatatypesforthemodernworld–XMLandJSON
XMLdatatypeandreturningdataasXMLfromfunctions
ReturningdataintheJSONformat
Summary
5.PL/pgSQLTriggerFunctions
Creatingthetriggerfunction
Creatingthetrigger
Workingonasimple“Hey,I’mcalled”trigger
Theaudittrigger
DisallowingDELETE
DisallowingTRUNCATE
ModifyingtheNEWrecord
Thetimestampingtrigger

Theimmutablefieldstrigger
Controllingwhenatriggeriscalled
Conditionaltriggers
Triggersonspecificfieldchanges
Visibility
Mostimportantly–usetriggerscautiously!
VariablespassedtothePL/pgSQLTRIGGERfunction
Summary
6.PostgreSQLEventTriggers
Usecasesforcreatingeventtriggers
Creatingeventtriggers
Creatinganaudittrail
Preventingschemachanges
Aroadmapofeventtriggers


Summary
7.DebuggingPL/pgSQL
ManualdebuggingwithRAISENOTICE
Throwingexceptions
Loggingtoafile
TheadvantagesofRAISENOTICE
ThedisadvantagesofRAISENOTICE
Visualdebugging
Installingthedebugger
Installingthedebuggerfromthesource
InstallingpgAdmin3
Usingthedebugger
Theadvantagesofthedebugger
Thedisadvantagesofthedebugger

Summary
8.UsingUnrestrictedLanguages
Areuntrustedlanguagesinferiortotrustedones?
Canyouuseuntrustedlanguagesforimportantfunctions?
Willuntrustedlanguagescorruptthedatabase?
Whyuntrusted?
WhyPL/Python?
QuickintroductiontoPL/Python
AminimalPL/Pythonfunction
Datatypeconversions
WritingsimplefunctionsinPL/Python
Asimplefunction
Functionsreturningarecord
Tablefunctions
Runningqueriesinthedatabase
Runningsimplequeries
Usingpreparedqueries


Cachingpreparedqueries
WritingtriggerfunctionsinPL/Python
Exploringtheinputsofatrigger
Alogtrigger
Constructingqueries
Handlingexceptions
AtomicityinPython
DebuggingPL/Python
Usingplpy.notice()totrackthefunction’sprogress
Usingassert
Redirectingsys.stdoutandsys.stderr

Thinkingoutofthe“SQLdatabaseserver”box
Generatingthumbnailswhensavingimages
Sendingane-mail
Listingdirectorycontents
Summary
9.WritingAdvancedFunctionsinC
ThesimplestCfunction–return(a+b)
add_func.c
Version0callconventions
Makefile
CREATEFUNCTIONadd(int,int)
add_func.sql.in
SummaryforwritingaCfunction
Addingfunctionalitytoadd(int,int)
SmarthandlingofNULLarguments
Workingwithanynumberofarguments
BasicguidelinesforwritingCcode
Memoryallocation
Usepalloc()andpfree()
Zero-fillthestructures


Includefiles
Publicsymbolnames
ErrorreportingfromCfunctions
“Error”statesthatarenoterrors
Whenaremessagessenttotheclient?
RunningqueriesandcallingPostgreSQLfunctions
AsampleCfunctionusingSPI
Visibilityofdatachanges

MoreinfoonSPI_*functions
Handlingrecordsasargumentsorreturnedvalues
Returningasingletupleofacomplextype
Extractingfieldsfromanargumenttuple
Constructingareturntuple
Interlude–whatisDatum?
Returningasetofrecords
Fastcapturingofdatabasechanges
Doingsomethingatcommit/rollback
Synchronizingbetweenbackends
WritingfunctionsinC++
AdditionalresourcesforC
Summary
10.ScalingYourDatabasewithPL/Proxy
Creatingasimplesingle-serverchat
Dealingwithsuccess–splittingtablesovermultipledatabases
Whatexpansionplansworkandwhen?
Movingtoabiggerserver
Master-slavereplication–movingreadstoslave
Multimasterreplication
Datapartitioningacrossmultipleservers
Splittingthedata
PL/Proxy–thepartitioninglanguage


InstallingPL/Proxy
ThePL/Proxylanguagesyntax
CONNECT,CLUSTER,andRUNON
SELECTandTARGET
SPLIT–distributingarrayelementsoverseveralpartitions

Thedistributionofdata
ConfiguringthePL/Proxyclusterusingfunctions
ConfiguringthePL/ProxyclusterusingSQL/MED
Movingdatafromthesingletothepartitioneddatabase
ConnectionPooling
Summary
11.PL/Perl–PerlProceduralLanguage
WhentousePL/Perl
InstallingPL/Perl
AsimplePL/Perlfunction
Passingandreturningnon-scalartypes
WritingPL/Perltriggers
UntrustedPerl
Summary
12.PL/Tcl–TclProceduralLanguage
InstallingPL/Tcl
AsimplePL/Tclfunction
NullcheckingwithStrictfunctions
Theparameterformat
Passingandreturningarrays
Passingcomposite-typearguments
Accessingdatabases
WritingPL/Tcltriggers
UntrustedTcl
Summary
13.PublishingYourCodeasPostgreSQLExtensions


Whentocreateanextension
Unpackagedextensions

Extensionversions
The.controlfile
Buildinganextension
Installinganextension
Viewingextensions
Publishingyourextension
IntroductiontoPostgreSQLExtensionNetwork
Signinguptopublishyourextension
Creatinganextensionprojecttheeasyway
Providingthemetadataabouttheextension
Writingyourextensioncode
Creatingthepackage
SubmittingthepackagetoPGXN
InstallinganextensionfromPGXN
Summary
14.PostgreSQLasanExtensibleRDBMS
Whatcan’tbeextended?
Creatinganewoperator
Overloadinganoperator
Optimizingoperators
COMMUTATOR
NEGATOR
Creatingindexaccessmethods
Creatinguser-definedaggregates
Usingforeigndatawrappers
Summary
Index

PostgreSQLServerProgrammingSecond
Edition



PostgreSQLServerProgrammingSecond
Edition
Copyright©2015PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,
ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthe
publisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyofthe
informationpresented.However,theinformationcontainedinthisbookissoldwithout
warranty,eitherexpressorimplied.Neithertheauthors,norPacktPublishing,andits
dealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecaused
directlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthe
companiesandproductsmentionedinthisbookbytheappropriateuseofcapitals.
However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:June2013
Secondedition:February2015
Productionreference:1210215
PublishedbyPacktPublishingLtd.
LiveryPlace
35LiveryStreet
BirminghamB32PB,UK.
ISBN978-1-78398-058-1
www.packtpub.com

Credits
Authors
UsamaDar
HannuKrosing
JimMlodgenski
KirkRoybal
Reviewers
StephenFrost
RickvanHattem
VibhorKumar
JeffLawson
MarianoReingart
JulienTachoires
CommissioningEditor
UshaIyer
AcquisitionEditors
AntonyLowe
MeetaRajani
SamWood
ContentDevelopmentEditor
AdrianRaposo
TechnicalEditors
MrunmayeePatil
ChinmayPuranik
CopyEditors
DiptiKapadia
AartiSaldanha
ProjectCoordinator
KinjalBari

Proofreaders
MariaGould


LindaMorris
Indexer
MonicaAjmeraMehta
ProductionCoordinator
NiteshThakur
CoverWork
NiteshThakur



AbouttheAuthors
UsamaDarisaseasonedsoftwaredeveloperandarchitect.Duringhis14years’career,he
hasworkedextensivelywithPostgreSQLandotherdatabasetechnologies.Heworkedon
PostgreSQLinternalsextensivelywhilehewasworkingforEnterpriseDB.Currently,he
livesinMunichwhereheworksforHuawei’sEuropeanResearchCenter.Hedesignsthe
nextgenerationofhigh-performancedatabasesystemsbasedonopensourcetechnologies,
suchasPostgreSQL,whichareusedunderhighworkloadsandstrictperformance
requirements.
HannuKrosingwasaPostgreSQLuserbeforeitwasrewrittentouseSQLasitsmain
querylanguagein1995.Therefore,hehasboththehistoricperspectiveofits
development,aswellasalmost20yearsofexperienceinusingittosolvevariousreal-life
problems.
HewasthefirstdatabaseadministratoranddatabasearchitectatSkype,whereheinvented
theshardinglanguagePL/Proxythatallowsyoutoscaletheuserdatabaseinordertowork
withbillionsofusers.
AfterheleftSkypeattheendof2006—aboutayearafteritwasboughtbyeBay—hehas

beenworkingasaPostgreSQLconsultantwith2ndQuadrant,thepremierPostgreSQL
consultancywithaglobalreachandlocalpresenceinmostpartsoftheworld.
HehascoauthoredPostgreSQL9AdministrationCookbook,PacktPublishing,together
withoneofthemainPostgreSQLdevelopers,SimonRiggs.
Iwanttosincerelythankmywife,Evelyn,forhersupportwhilewritingthisbook.
JimMlodgenskiistheCTOofOpenSCG,aprofessionalservicescompanyfocusedon
leveragingopensourcetechnologiesforstrategicadvantage.HewasformerlytheCEOof
StormDB,adatabasecloudcompanyfocusedonhorizontalscalability.PriortoStormDB,
hehasheldhighlytechnicalrolesatCirrusTechnology,Inc.,EnterpriseDB,andFusion
Technologies.
JimisalsoaferventadvocateofPostgreSQL.HeisontheboardoftheUnitedStates
PostgreSQLAssociationaswellasapartoftheorganizingteamsoftheNewYork
PostgreSQLUserGroupandPhiladelphiaPostgreSQLUserGroup.
KirkRoybalhasbeenanactivememberofthePostgreSQLcommunitysince1998.He
hashelpedorganizeusergroupsinHouston,Dallas,andBloomington,IL.Hehas
mentoredmanyjuniordatabaseadministratorsandprovidedcross-trainingtosenior
databaseengineers.HehasprovidedsolutionsusingPostgreSQLforreporting,business
intelligence,datawarehousing,applications,anddevelopmentsupport.
HesawthescopeofPostgreSQLwhenhisfirstsmall-scalebusinesscustomeraskedfora
webapplication.Atthattime,competitivedatabaseproductswereeitherextremely
immatureorcostprohibitive.
KirkhasstoodbyhischoiceofPostgreSQLformanyyearsnow.Hisexpertiseisfounded
onkeepingupwithfeaturesandcapabilitiesastheybecomeavailable.


Writingabookhasbeenauniqueexperienceforme.Manypeoplefantasizeaboutit,few
startone,andevenfewergettopublication.Iamproudtobepartofateamthatactually
madeittothebookshelf(whichitselfisadiminishingbreed).ThankstoSarahCullington
fromPacktPublishingforgivingmeachancetoparticipateintheproject.Ibelievethat
thePostgreSQLcommunitywillbebetterservedbythisinformation,andIhopethatthey

receivethisasarewardforthetimethattheyhaveinvestedinmeovertheyears.
Abookonlyhasthevaluethatthereadersgiveit.ThankyoutothePostgreSQL
communityforallthetechnical,personal,andprofessionaldevelopmenthelpyouhave
provided.ThePostgreSQLcommunityisagreatbunchofpeople,andIhaveenjoyedthe
companyofmanyofthem.Ihopetocontributemoretothisprojectinthefuture,andI
hopeyoufindmycontributionsasvaluableasIfindyours.
Thankyoutomyfamilyforgivingmeareasontosucceedandforlisteningtothe
gobbledygookandnoddingappreciatively.
Haveyoueverhadyourfamilyaskyouwhatyouweredoingandansweredthemwitha
function?Tryit.No,thenagain,don’ttryit.Theymayjusthaveyouinvoluntarily
checkedinsomewhere.



AbouttheReviewers
StephenFrostisamajorcontributorandcommittertoPostgreSQL,whohasbeen
involvedwithPostgreSQLsince2002,andhasdevelopedfeaturessuchastherolesystem
andcolumn-levelprivileges.
HeisthechieftechnologyofficeratCrunchyDataSolutions,Inc.,thePostgreSQL
companyforSecureEnterprises.HeisinvolvedintheadvancementofPostgreSQL’s
capabilities,particularlyintheareaofsecurityinordertosupporttheneedsofgovernment
andfinancialinstitutionswhohavestrictsecurityandregulatoryrequirements.
RickvanHattemisanentrepreneurwithacomputersciencebackgroundandalong-time
opensourcedeveloperwithvastexperienceintheC,C++,Python,andJavalanguages.
Additionally,hehasworkedwithmostlargedatabaseserverssuchasOracle,MSSQL,
andMySQL,buthehasbeenfocusingonPostgreSQLsinceVersion7.4.
HeisoneofthefoundersoftheFashiolista.comsocialnetwork,anduntilrecently,hewas
theCTO.Here,heusedPostgreSQLtoscalethefeedsformillionsofuserstoshowthat
PostgreSQLcanholduptoNoSQLsolutions,givensometuningandadditionaltools.
AfterFashiolista,heworkedasafreelanceconsultantforseveralcompanies,including

2ndQuadrant.
HeiscurrentlythefounderofPGMon.com,amonitoringservicethatanalyzesyour
databases,indexes,andqueriestokeepthemrunningatpeakperformance.Inadditionto
analyzingyourdatabasesettings,thesystemactivelymonitorsyourqueriesandgivesyou
recommendationstoenhanceperformance.
Heisalsothecreatorandmaintainerofalargenumberofopensourceprojects,suchas
pg_query_analyser,pg_cascade_timestamp,QtQuery,Python-Statsd,andDjango-Statsd.
VibhorKumarisaprincipalsystemarchitectatEnterpriseDBwhospecializesin
assistingFortune100companiestodeploy,manage,andoptimizePostgresdatabases.He
joinedEnterpriseDBin2008toworkwithPostgresafterseveralyearsofworkingwith
Oraclesystems.HehasworkedinteamleadershiprolesatIBMGlobalServicesandBMC
SoftwareaswellasanOracledatabaseadministratoratCMCLtd.forseveralyears.He
hasdevelopedexpertiseinOracle,DB2,andMongoDBandholdscertificationsinthem.
HehasexperienceworkingwithMSSQLServer,MySQL,anddatawarehousing.He
holdsabachelor’sdegreeincomputersciencefromtheUniversityofLucknowanda
master’sdegreeincomputersciencefromtheArmyInstituteofManagement,Kolkata.He
isacertifiedPostgreSQLtrainerandholdsaprofessionalcertificationinPostgresPlus
AdvancedServerfromEnterpriseDB.
JeffLawsonhasbeenafananduserofPostgreSQLsincethetimehediscovereditin
2001.Overtheyears,hehasalsodevelopedanddeployedapplicationsforIBMDB2,
Oracle,MySQL,MicrosoftSQLServer,Sybase,andothers,buthealwaysprefers
PostgreSQLforitsbalanceoffeaturesandopenness.Muchofhisexperienceinvolves
developingforInternet-facingwebsites/projectsthatrequirehighlyscalabledatabases
withhighavailabilityorwithprovisionsfordisasterrecovery.


HecurrentlyworksasthedirectorofsoftwaredevelopmentforFlightAware,whichisan
airplane-trackingwebsitethatusesPostgreSQLandotheropensourcesoftwaretostore
andanalyzethepositionsofthethousandsofflightsthatareoperatedworldwideevery
day.Hehasextensiveexperienceinsoftwarearchitecture,datasecurity,andnetwork

protocoldesignfromthesoftwareengineeringpositionshehasheldatUniva/United
Devices,Microsoft,NASA’sJetPropulsionLaboratory,andWolfeTech.Heisafounderof
distributed.net,whichpioneereddistributedcomputinginthe1990s,andhecontinuesto
serveasthechiefofoperationsandasamemberoftheboardthere.HeearnedaBSc
degreeincomputersciencefromHarveyMuddCollege.
Heisfondofcattle,holdsanFAAprivatepilotcertificate,andownsanairplanebasedin
Houston,Texas.
MarianoReingartlivesinBuenosAires,Argentina,andisaspecialistinthesoftware
developmentofapplicationsandlibraries(webservices,PDF,GUI,replication,andsoon)
withmorethan10yearsofexperience.Currently,heisthePostgreSQLregionalcontact
forArgentinaandaPythonSoftwareFoundationmember.
Heisamajorcontributortotheweb2pyPythonwebframework,andnowhe’sworkingon
thewxWidgetsmultiplatformGUItoolkit(specificallyintheQtportandAndroidmobile
areas).Also,hehascontributedtomorethanadozenopensourceprojects,includingan
interfaceforFreeElectronicInvoicewebservices(PyAfipWs)andPythonicreplication
forPostgreSQL(PyReplica).
Hehasabachelor’sdegreeincomputersystemsanalysis,andcurrently,he’samaster’s
candidatefortheMScinfreesoftwaredegreeattheOpenUniversityofCatalonia.
Heworksonhisownfundedentrepreneurialventureformedbyanopengroupof
independentprofessionals,dedicatedtosoftwaredevelopment,training,andtechnical
support,focusingonopensourcetools(GNU/Linux,Python,PostgreSQL,and
web2py/wxPython).
HehasworkedforlocalPython-basedcompaniesinlargebusinessapplications(ERP,
SCM,andCRM)andmissioncriticalsystems(electioncounting,electronicvoting,and
911emergencyeventssupport).Hehascontributedtobookssuchasweb2pyEnterprise
WebFramework,ThirdEdition,andweb2pyApplicationDevelopmentCookbook,Packt
Publishing,andseveralSpanishtranslationsofthePostgreSQLofficialdocumentation.
Hisfullresumeisavailableat />JulienTachoiresisaPostgreSQLspecialist,whoworksasconsultantfortheFrench
PostgreSQLcompanyDalibo.Heisthemaindeveloperofpg_activity,atop-endsoftware
dedicatedtofollowthePostgreSQLincomingtrafficinrealtime,whichiswrittenin

Python.
IwanttothankmyemployerDalibo;mywife,Camille;andmyson,Arthur.