IT training whats new in ecto 2 0 1 khotailieu

TableofContents

Foreword
Introduction
1.EctoisnotyourORM
2.Schemalessqueries
3.Schemalesschangesets
4.Dynamicqueries
5.Multitenancywithqueryprefixes
6.Aggregatesandsubqueries
7.Improvedassociationsandfactories
8.Manytomanyandcasting
9.Manytomanyandupserts
10.ComposabletransactionswithEcto.Multi
11.ConcurrenttestswiththeSQLSandbox

2


Foreword

Foreword

InJanuary2017,wewillcelebrate5yearssincewedecidedtoinvestinElixir.Backin2012,
JoséValim,ourco-founderandpartner,presentedustheideaofaprogramminglanguage
thatwouldbeexpressive,embraceproductivityinitstooling,andleveragetheErlangVMto
notonlytackletheproblemsinwritingconcurrentsoftwarebutalsotobuildfault-tolerantand
distributedsystems.
Elixircontinued,insomesense,tobeariskyprojectformonths.Wewerecertainlyexcited
aboutspreadingfunctional,concurrentanddistributedprogrammingconceptstomoreand

moredevelopers,hopingitwouldleadtoapositiveimpactonthesoftwaredevelopment
industry,butdevelopingalanguageisalong-termeffortthatmayneverbecomeconcrete.
Duringthesummerof2013,othercompaniesanddevelopersstartedtoshowintereston
Elixir.Weheardaboutcompaniesusingitinproduction,moredevelopersbeganto
contributeandcreatetheirownprojects,differentpublisherswerewritingbooksonthe
language,andsoon.SucheventsgaveustheconfidencetoinvestmoreinElixirandbring
thelanguagetoversion1.0.
OnceElix-ir1.0waslaunchedinSeptember2014,weturnedourfocustothewebplatform.
WetidiedupPlug,thebuildingblockforwritingwebapplicationsinElixir.Wealsofocused
intensivelyonEcto,bringingittoversion1.0togetherwiththeEctoteam,andthenworked
alongsideChrisMcCordandteamtogetthefirstmajorPhoenixreleaseout.Duringthistime
wealsostartedother
Today,boththecommunityandouropensourceprojectsareshowingsteadyandhealthy
growth.Elixirisastablelanguagewithcontinuousimprovementslandedinminorversions.
PlugcontinuestobeasolidfoundationforframeworkssuchasPhoenix.Ecto,however,
requiredmorethanasmallnudgeintherightdirection.Werealizedthatweneededtoletgo
ofold,harmfulhabitsandmakeEctolessofanabstractionlayerandmoreofatoolyou
controlandapplytodifferentproblems.

3


Foreword

ThisbookisthefinaleffortbehindEcto2.0.Itshowcasesthenewdirectionwehaveplanned
forEcto,thestructuralimprovementsmadebytheEctoteamandmanyofitsnewfeatures.
Wehopeyouwillenjoyit.Afterall,itistimetoletgoofpasthabits.
Havefun,
-ThePlataformatecteam

4


Foreword

CONTACTUS

5


Introduction

Introduction

Ecto2.0isasubstantialdeparturefromearlierversions.Insteadofthinkingaboutmodels,
Ecto2.0aimstoprovidedevelopersawiderangeofdata-centrictools.Therefore,inorderto
useEcto2.0effectively,wemustlearnhowtowieldthosetoolsproperly.That'sthegoalof
thisbook.
Thisbook,however,isnotanintroductiontoEcto.IfyouhaveneverusedEctobefore,we
recommendyoutogetstartedwithEcto'sdocumentationandlearnmoreaboutrepositories,
queries,schemasandchangesets.Weassumethereaderisfamiliarwiththesebuilding
blocksandhowtheyrelatetoeachother.
ThefirstchaptersofthebookwillcoverthebiggestconceptualchangesinEcto2.0.Wewill
talkaboutrelationalmappersin"EctoisnotyourORM"andthenexploreSchemaless
QueriesandtherelationshipbetweenSchemasandChangesets.
Afterwewilltakeadeeperlookintoqueries,discussinghowEcto2.0makesiteasierto
builddynamicqueries,howtotargetdifferentdatabasesviaqueryprefixes,aswellasthe
newaggregateandsubqueryfeatures.
Thenwewillgobacktoschemasanddiscusstheschema-relatedenhancementsthatare
partofEcto2.0,suchastheimprovedassociationsupport, many_to_manyassociationsand

Ecto's2.1upsertsupport.
Finally,wewillexplorebrandnewtopics,likethenewEctoSQLSandbox,thatallows
developerstoruntestsagainstthedatabaseconcurrently,aswellas Ecto.Multi,which
makesworkingwithtransactionssimplerthanever.

Acknoledgements

6


Introduction

WewanttothanktheEctoteamfortheirfantasticworkbehindEcto:EricMeadows-Jönsson,
JamesFish,JoséValimandMichałMuskała.Wealsothankeveryonewhohascontributed
toEcto,beitwithcode,documentation,bywritingarticles,givingpresentations,organizing
workshops,etc.
Finallyweappreciateeveryonewhohasreviewedourbetaeditionsandsentusfeedback:
AdamRutkowski,AlkisTsamis,ChristianvonRoques,CurtisEkstrom,EricMeadowsJönsson,KevinBaird,KevinRankin,MichaelMadrid,MichałMuskała,RaphaelVidal,Steve
Pallen,TobiasPfeifferandWojtekMach.

7


1.EctoisnotyourORM

EctoisnotyourORM

Dependingonyourperspective,thisisaratherboldorobviousstatementtostartthisbook.
Afterall,Elixirisnotanobject-orientedlanguage,soEctocan'tbeanObject-relational
Mapper.However,thisstatementisslightlymorenuancedthanitlooksandthereare

importantlessonstobelearnedhere.

OisforObjects
Atitscore,objectscouplestateandbehaviourtogether.Inthesame userobject,youcan
havedata,likethe user.name,aswellasbehaviour,likeconfirmingaparticularuser
accountvia user.confirm().Whilesomelanguagesenforcedifferentsyntaxesbetween
accessingdata( user.namewithoutparentheses)andbehaviour( user.confirm()with
parentheses),otherlanguagesfollowtheUniformAccessPrincipleinwhichanobjectshould
notmakeadistinctionbetweenthetwosyntaxes.EiffelandRubyarelanguagesthatfollow
suchprinciple.
Elixirfailsthe"couplingofstateandbehaviour"test.InElixir,weworkwithdifferentdata
structuressuchastuples,lists,mapsandothers.Behaviourcannotbeattachedtodata
structures.Behaviourisalwaysaddedtomodulesviafunctions.
Whenthereisaneedtoworkwithstructureddata,Elixirprovidesstructs.Structsdefinea
setoffields.Astructwillbereferencedbythenameofthemodulewhereitisdefined:
defmoduleUserdo
defstruct[:name,:email]
end
user=%User{name:"JohnDoe",email:""}

8


1.EctoisnotyourORM

Onceauserstructiscreated,wecanaccessitsemailvia user.email.However,structsare
onlydata.Itisimpossibletoinvoke user.confirm()onaparticularstructinawayitwill
executecoderelatedtoe-mailconfirmation.
Althoughwecannotattachbehaviourtostructs,itispossibletoaddfunctionstothesame
modulethatdefinesthestruct:

defmoduleUserdo
defstruct[:name,:email]
defconfirm(user)do
#Confirmtheuseremail
end
end

Evenwiththedefinitionabove,itisimpossibleinElixirtoconfirmagivenuserbycalling
user.confirm().Instead,the Userprefixisrequiredandthe userstructmustbeexplicitly

givenasargument,asin User.confirm(user).Attheendoftheday,thereisnostructural
couplingbetweenthe userstructandthefunctionsinthe Usermodule.HenceElixirdoes
nothavemethods,ithasfunctions.
Withouthavingobjects,Ectocertainlycan'tbeanORM.However,ifweletgooftheletter
"O"forasecond,canEctostillbearelationalmapper?

Relationalmappers
AnObject-RelationalMapperisatechniqueforconvertingdatabetweenincompatibletype
systems,commonlydatabases,toobjects,andback.
Similarly,EctoprovidesschemasthatmapsanydatasourceintoanElixirstruct.When
appliedtoyourdatabase,Ectoschemasarerelationalmappers.Therefore,whileEctoisnot
arelationalmapper,itcontainsarelationalmapperaspartofthemanydifferenttoolsit
offers.
Forexample,theschemabelowtiesthefields name, email, inserted_atand
updated_attofieldssimilarlynamedinthe userstable:

9


1.EctoisnotyourORM

defmoduleMyApp.Userdo
useEcto.Schema
schema"users"do
field:name
field:email
timestamps()
end
end

Theappealbehindschemasisthatyoudefinetheshapeofthedataonceandyoucanuse
thisshapetoretrievedatafromthedatabaseaswellascoordinatechangeshappeningon
thedata:
MyApp.User
|>MyApp.Repo.get!(13)
|>Ecto.Changeset.cast([name:"newname"],[:name,:email])
|>MyApp.Repo.update!

Byrelyingontheschemainformation,Ectoknowshowtoreadandwritedatawithoutextra
inputfromthedeveloper.Insmallapplications,thiscouplingbetweenthedataandits
representationisdesired.However,whenusedwrongly,itleadstocomplexcodebasesand
subparsolutions.
ItisimportanttounderstandtherelationshipbetweenEctoandrelationalmappersbecause
saying"EctoisnotyourORM"doesnotautomaticallysaveEctoschemasfromsomeofthe
downsidesmanydevelopersassociateORMswith.
HerearesomeexamplesofissuesoftenassociatedwithORMsthatEctodevelopersmay
runintowhenusingschemas:
ProjectsusingEctomayend-upwith"GodSchemas",commonlyreferredas"God
Models","FatModels"or"CanonicalModels"insomelanguagesandframeworks.Such
schemascouldcontainhundredsoffields,oftenreflectingbaddecisionsdoneatthe

datalayer.Insteadofprovidingonesingleschemawithfieldsthatspanmultiple
concerns,itisbettertobreaktheschemaacrossmultiplecontexts.Forexample,
insteadofasingle MyApp.Userschemawithdozensoffields,considerbreakingitinto
MyApp.Accounts.User, MyApp.Purchases.Userandsoon.Eachstructwithfields

exclusivetoitsenclosingcontext
Developersmayexcessivelyrelyonschemaswhensometimesthebestwaytoretrieve
datafromthedatabaseisintoregulardatastructures(likemapsandtuples)andnot
pre-definedshapesofdatalikestructs.Forexample,whendoingsearches,generating

10


1.EctoisnotyourORM

reportsandothers,thereisnoreasontorelyorreturnschemasfromsuchqueries,asit
oftenreliesondatacomingfrommultipletableswithdifferentrequirements
Developersmaytrytousethesameschemaforoperationsthatmaybequitedifferent
structurally.Manyapplicationswouldboltfeaturessuchasregistration,accountlogin,
intoasingleUserschema,whilehandlingeachoperationindividually,possiblyusing
differentschemas,wouldleadtosimplerandclearersolutions
Inthenexttwochapters,wewanttobreakthose"badpractices"apartbyexploringhowto
useEctowithnoormultipleschemaspercontext.Bylearninghowtoinsert,delete,
manipulateandvalidatedatawithandwithoutschemas,wehopedeveloperswillfeel
comfortablewithbuildingcomplexapplicationswithoutrelyingonone-size-fits-allschemas.

11


2.Schemalessqueries

Schemalessqueries

MostqueriesinEctoarewrittenusingschemas.Forexample,toretrieveallpostsina
database,onemaywrite:
MyApp.Repo.all(Post)

Intheconstructabove,Ectoknowsallfieldsandtheirtypesintheschema,rewritingthe
queryaboveto:
MyApp.Repo.all(frompinPost,select:%Post{title:p.title,body:p.body,...})

Interestingly,backinEcto'searlydays,therewasnosuchthingasschemas.Queriescould
onlybewrittendirectlyagainstadatabasetablebypassingthetablenameasastring:
MyApp.Repo.all(frompin"posts",select:{p.title,p.body})

Whenwritingschemalessqueries,theselectexpressionmustbeexplicitlywrittenwithallthe
desiredfields.
WhiletheabovesyntaxmadeitintoEcto1.0,bythetimeEcto1.0waslaunched,mostof
thedevelopmentfocusinEctohadchangedtowardsschemas.Thismeanswhiledevelopers
wereabletoreaddatawithoutschemas,theywereoftentooverbose.Notonlythat,ifyou
wantedtoinsertentriestoyourdatabasewithoutschemas,youwereoutofluck.
Ecto2.0levelsupthegamebyaddingmanyimprovementstoschemalessqueries,notonly
byimprovingthesyntaxforreadingandupdatingdata,butalsobyallowingalldatabase
operationstobeexpressedwithoutaschema.

12


2.Schemalessqueries

insert_all
OneofthefunctionsaddedtoEcto2.0is Ecto.Repo.insert_all/3.With insert_all,
developerscaninsertmultipleentriesatonceintoarepository:
MyApp.Repo.insert_all(Post,[[title:"hello",body:"world"],
[title:"another",body:"post"]])

Although insert_allisjustaregularElixirfunction,itplaysanimportantroleinEcto2.0as
itallowsdeveloperstoread,create,updateanddeleteentrieswithoutaschema.
insert_allwasthelastpieceofthepuzzle.Let'sseesomeexamples.

Ifyouarewritingareportingview,itmaybecounter-productivetothinkhowyourexisting
applicationschemasrelatetothereportbeinggenerated.Itisoftensimplertowriteaquery
thatreturnsonlythedatayouneed,withouttryingtofitthedataintoexistingschemas:
importEcto.Query
defrunning_activities(start_at,end_at)
MyApp.Repo.all(
fromuin"users",
join:ain"activities",
on:a.user_id==u.id,
where:a.start_at>type(^start_at,Ecto.DateTime)and
a.end_atgroup_by:a.user_id,
select:%{user_id:a.user_id,interval:a.start_at-a.end_at,count:count(u.id
)}
)
end

Thefunctionabovedoesnotrelyonschemas.Itreturnsonlythedatathatmattersfor
buildingthereport.Noticehowweusethe type/2functiontospecifywhatistheexpected
typeoftheargumentweareinterpolating,benefitingfromthesametypecastingguarantees

aschemawouldgive.
Inserts,updatesanddeletescanalsobedonewithoutschemasvia insert_all,
update_alland delete_allrespectively:

13


2.Schemalessqueries

#InsertdataintopostsandreturnitsID
[%{id:id}]=
MyApp.Repo.insert_all"posts",[[title:"hello"]],returning:[:id]
#UsetheIDtotriggerupdates
post=frompin"posts",where:[id:^id]
{1,_}=MyApp.Repo.update_allpost,set:[title:"newtitle"]
#Aswellasfordeletes
{1,_}=MyApp.Repo.delete_allpost

ItisnothardtoseehowtheseoperationsdirectlymaptotheirSQLvariants,keepingthe
databaseatyourfingertipswithouttheneedtointermediatealloperationsthroughschemas.

Simplerqueries
Besidessupportingschemalessinserts,updatesanddeletesqueries,withvaryingdegrees
ofcomplexity,Ecto2.0alsomakesregularschemalessqueriesmoreexpressive.
Oneexampleistheabilitytoselectalldesiredfieldswithoutduplication.Inearlyversions,
youwouldhavetowriteverboseselectexpressionssuchas:
frompin"posts",select:%{title:p.title,body:p.body}

WithEcto2.0youcansimplypassthedesiredlistoffieldsdirectly:
from"posts",select:[:title,:body]

Thetwoqueriesaboveareequivalent.Whenalistoffieldsisgiven,Ectowillautomatically
convertthelistoffieldstoamaporastruct.
Supportforpassingalistoffieldsorkeywordlistshasbeenaddedtoalmostallquery
constructsinEcto2.0.Forexample,wecanuseanupdatequerytochangethetitleofa
givenpostwithoutaschema:
defupdate_title(post,new_title)do
query=from"posts",where:[id:^post.id],update:[set:[title:^new_title]]
MyApp.Repo.update_all(query)
end

The updateconstructsupportsfourcommands:
:set-setsthegivencolumntothegivenvalues

14


2.Schemalessqueries

:inc-incrementsthegivencolumnbythegivenvalue
:push-pushes(appends)thegivenvaluetotheendofanarraycolumn
:pull-pulls(removes)thegivenvaluefromanarraycolumn

Forexample,wecanincrementacolumnatomicallybyusingthe :inccommand,withor
withoutschemas:
defincrement_page_views(post)do
query=from"posts",where:[id:^post.id],update:[inc:[page_views:1]]
MyApp.Repo.update_all(query)
end

Byallowingregulardatastructurestobegiventomostqueryoperations,Ecto2.0makes
querieswithandwithoutschemasmoreaccessible.Notonlythat,italsoenablesdevelopers
towritedynamicqueries,wherefields,filters,orderingcannotbespecifiedupfront.Wewill
exploresuchwithmoredetailsinupcomingchapters.Fornow,let'scontinueexploring
schemasinthecontextofchangesets.

15


3.Schemalesschangesets

Schemasandchangesets

Inthelastchapterwelearnedhowtoperformalldatabaseoperations,frominsertionto
deletion,withoutusingaschema.Whilewehavebeenexploringtheabilitytowrite
constructswithoutschemas,wehaven'tdiscussedwhatschemasactuallyare.Inthis
chapter,wewillrectifythat.
Inthischapterwewilltakealookattheroleschemasplaywhenvalidatingandcastingdata
throughchangesets.Aswewillsee,sometimesthebestsolutionisnottocompletelyavoid
schemas,butbreakalargeschemaintosmallerones.Maybeoneforreadingdata,another
forwriting.Maybeoneforyourdatabase,anotherforyourforms.

Schemasaremappers
TheEctodocumentationsays:
AnEctoschemaisusedtomapanydatasourceintoanElixirstruct.
WeputemphasisonanybecauseitisacommonmisconceptiontothinkEctoschemasmap
onlytoyourdatabasetables.
Forinstance,whenyouwriteawebapplicationusingPhoenixandyouuseEctotoreceive
externalchangesandapplysuchchangestoyourdatabase,wehavethismapping:
Database<->Ectoschema<->Forms/API

AlthoughthereisasingleEctoschemamappingtobothyourdatabaseandyourAPI,in
manysituationsitisbettertobreakthismappingintwo.Let'sseesomepracticalexamples.

16


3.Schemalesschangesets

Imagineyouareworkingwithaclientthatwantsthe"SignUp"formtocontainthefields
"Firstname","Lastname"alongside"E-mail"andotherinformation.Youknowtherearea
coupleproblemswiththisapproach.
Firstofall,noteveryonehasafirstandlastname.Althoughyourclientisdecidedon
presentingbothfields,theyareaUIconcern,andyoudon'twanttheUItodictatetheshape
ofyourdata.Furthermore,youknowitwouldbeusefultobreakthe"SignUp"information
acrosstwotables,the"accounts"and"profiles"tables.
Giventherequirementsabove,howwouldweimplementtheSignUpfeatureinthe
backend?
Oneapproachwouldbetohavetwoschemas,AccountandProfile,withvirtualfieldssuch
as first_nameand last_name,anduseassociationsalongsidenestedformstotiethe
schemastoyourUI.Oneofsuchschemaswouldbe:
defmoduleProfiledo
useEcto.Schema
schema"profiles"do
field:name
field:first_name,:string,virtual:true
field:last_name,:string,virtual:true
...
end
end

ItisnothardtoseehowwearepollutingourProfileschemawithUIrequirementsbyadding
fieldssuch first_nameand last_name.IftheProfileschemaisusedforbothreadingand
writingdata,itmayend-upinanawkwardplacewhereitisnotusefulforany,asitcontains
fieldsthatmapjusttooneortheotheroperation.
Onealternativesolutionistobreakthe"Database<->Ectoschema<->Forms/API"
mappingintwoparts.Thefirstwillcastandvalidatetheexternaldatawithitsownstructure
whichyouthentransformandwritetothedatabase.Forsuch,let'sdefineaschemanamed
Registrationthatwilltakecareofcastingandvalidatingtheformdataexclusively,mapping

directlytotheUIfields:

17


3.Schemalesschangesets

defmoduleRegistrationdo
useEcto.Schema
embedded_schemado
field:first_name
field:last_name
field:email
end
end

Weused embedded_schemabecauseitisnotourintenttopersistitanywhere.Withthe
schemainhand,wecanuseEctochangesetsandvalidationstoprocessthedata:
fields=[:first_name,:last_name,:email]
changeset=

%Registration{}
|>Ecto.Changeset.cast(params["sign_up"],fields)
|>validate_required(...)
|>validate_length(...)

Nowthattheregistrationchangesaremappedandvalidated,wecancheckiftheresulting
changesetisvalidandactaccordingly:
ifchangeset.valid?do
#Getthemodifiedregistrationstructoutofthechangeset
registration=Ecto.Changeset.apply_changes(changeset)
MyApp.Repo.transactionfn->
MyApp.Repo.insert_all"accounts",[Registration.to_account(registration)]
MyApp.Repo.insert_all"profiles",[Registration.to_profile(registration)]
end
{:ok,registration}
else
#AnnotatetheactionwetriedtoperformsotheUIshowserrors
changeset=%{changeset|action::registration}
{:error,changeset}
end

The to_account/1and to_profile/1functionsin Registrationwouldreceivethe
registrationstructandsplittheattributesapartaccordingly:

18


3.Schemalesschangesets

defto_account(registration)do

Map.take(registration,[:email])
end
defto_profile(%{first_name:first,last_name:last})do
%{name:"#{first}#{last}"}
end

Intheexampleabove,bybreakingapartthemappingbetweenthedatabaseandElixirand
betweenElixirandtheUI,ourcodebecomesclearerandourdatastructuressimpler.
Notewehaveused MyApp.Repo.insert_all/2toadddatatoboth"accounts"and"profiles"
tablesdirectly.Wehavechosentobypassschemasaltogether.However,thereisnothing
stoppingyoufromalsodefiningboth Accountand Profileschemasandchanging
to_account/1and to_profile/1torespectivelyreturn %Account{}and %Profile{}

structs.Oncestructsarereturned,theycouldbeinsertedthroughtheusual
MyApp.Repo.insert/2operation.Doingsocanbeespeciallyusefulifthereareuniquenessor

otherconstraintsthatyouwanttocheckduringinsertion.

Schemalesschangesets
Althoughwechosetodefinea Registrationschematouseinthechangeset,Ecto2.0also
allowsdeveloperstousechangesetswithoutschemas.Wecandynamicallydefinethedata
andtheirtypes.Let'srewritetheregistrationchangesetabovetobypassschemas:
data=%{}
types=%{first_name::string,last_name::string,email::string}
changeset=
{data,types}#Thedata+typestupleisequivalentto%Registration{}
|>Ecto.Changeset.cast(params["sign_up"],Map.keys(types))
|>validate_required(...)
|>validate_length(...)

YoucanusethistechniquetovalidateAPIendpoints,searchforms,andothersourcesof
data.Thechoiceofusingschemasdependsmostlyifyouwanttousethesamemappingin
differentplacesorifyoudesirethecompile-timeguaranteesElixirstructsgivesyou.
Otherwise,youcanbypassschemasaltogether,beitwhenusingchangesetsorinteracting
withtherepository.
However,themostimportantlessoninthischapterisnotwhentouseornottouse
schemas,butratherunderstandwhenabigproblemcanbebrokenintosmallerproblems
thatcanbesolvedindependentlyleadingtoanoverallcleanersolution.Thechoiceofusing

19


3.Schemalesschangesets

schemasornotabovedidn'taffectthesolutionasmuchasthechoiceofbreakingthe
registrationproblemapart.

20


4.Dynamicqueries

Dynamicqueries

EctowasdesignedfromthegrounduptohaveanexpressivequeryAPIthatleveragesElixir
syntaxtowritequeriesthatarepre-compiledforperformanceandsafety.Whenbuilding
queries,wemayusethekeywordssyntax
importEcto.Query
frompinPost,
where:p.author=="José"andp.category=="Elixir",

where:p.published_at>^minimum_date,
order_by:[desc:p.published_at]

orthepipe-basedone
importEcto.Query
Post
|>where([p],p.author=="José"andp.category=="Elixir")
|>where([p],p.published_at>^minimum_date)
|>order_by([p],desc:p.published_at)

Whilemanydeveloperspreferthepipe-basedsyntax,havingtorepeatthebinding pmade
itquiteverbosecomparedtothekeywordone.Furthermore,thecompile-timeaspectofEcto
querieswasatoddswithbuildingqueriesdynamically.
Imagineforexampleawebapplicationthatprovidessearchfunctionalityontopofexisting
posts.Theusershouldbeabletospecifymultiplecriteria,suchastheauthorname,thepost
category,publishinginterval,etc.
InEcto1.0,theonlywaytowritesuchfunctionalitywouldbevia Enum.reduce/3:

21


4.Dynamicqueries

deffilter(params)do
Enum.reduce(params,Post,&filter/2)
end
defpfilter({"author",author},query)do
where(query,[p],p.author==^author)
end
defpfilter({"category",category},query)do

where(query,[p],p.category==^category)
end
defpfilter({"published_at",minimum_date},query)do
where(query,[p],p.published_at>^minimum_date)
end
defpfilter({"order_by","published_at_asc"},query)do
order_by(query,[p],asc:p.published_at)
end
defpfilter({"order_by","published_at_desc"},query)do
order_by(query,[p],desc:p.published_at)
end
defpfilter(_ignore_unknown,query)do
query
end

Whilethecodeaboveworksfine,itcouplestheprocessingoftheparameterswiththequery
generation.Itisaverboseimplementationthatisalsohardtotestsincetheresultoffiltering
andhandlingofparametersarestoreddirectlyinthequerystruct.
Abetterapproachwouldbetoprocesstheparametersintoregulardatastructuresandthen
buildthequeryaslateaspossible.That'sexactlywhatEcto2.0allowsustodo.

Focusingondatastructures
Ecto2.0providesasimplerAPIforbothkeywordandpipebasedqueriesbymakingdata
structuresfirst-class.Let'srewritetheoriginalqueriestousedatastructureswhenpossible:
frompinPost,
where:[author:"José",category:"Elixir"],
where:p.published_at>^minimum_date,
order_by:[desc::published_at]

and

22


4.Dynamicqueries

Post
|>where(author:"José",category:"Elixir")
|>where([p],p.published_at>^minimum_date)
|>order_by(desc::published_at)

Noticehowwewereabletoditchthe pselectorinmostexpressions.InEcto2.0,all
constructs,from selectand order_byto whereand group_by,acceptdatastructuresas
input.Thedatastructurecanbespecifiedatcompile-time,asabove,andalsodynamicallyat
runtime,shownbelow:
where=[author:"José",category:"Elixir"]
order_by=[desc::published_at]
Post
|>where(^where)
|>where([p],p.published_at>^minimum_date)
|>order_by(^order_by)

Theadvantageofinterpolatingdatastructuresisthatwecandecoupletheprocessingof
parametersfromthequerygeneration.Notehowevernotallexpressionscanbeconverted
todatastructures.Since whereconvertsakey-valuetoa key==valuecomparison,orderbasedcomparisonssuchas p.published_at>^minimum_datestillneedtobewrittenaspart
ofthequeryasbefore.
Luckily,Ecto2.1solvesthisissue.

Thedynamicmacro
Forcaseswherewecannotrelyondatastructuresbutstilldesiretobuildqueries

dynamically,Ecto2.1includesthe Ecto.Query.dynamic/2macro.
Inordertounderstandhowthe dynamicmacroworkslet'srewritethe filter/1function
fromthebeginningofthischapterusingbothdatastructuresandthe dynamicmacro.The
examplebelowrequiresEcto2.1:

23


4.Dynamicqueries

deffilter(params)do
Post
|>order_by(^filter_order_by(params["order_by"]))
|>where(^filter_where(params))
|>where(^filter_published_at(params["published_at"]))
end
deffilter_order_by("published_at_desc"),do:[desc::published_at]
deffilter_order_by("published_at"),do:[asc::published_at]
deffilter_order_by(_),do:[]
deffilter_where(params)do
forkey<-[:author,:category],
value=params[Atom.to_string(params)],
do:{key,value}
end
deffilter_published_at(date)whenis_binary(date),
do:dynamic([p],p.published_at>^date)
deffilter_published_at(_date),
do:true

The dynamicmacroallowsustobuilddynamicexpressionsthatarelaterinterpolatedinto

thequery. dynamicexpressionscanalsobeinterpolatedintodynamicexpressions,allowing
developerstobuildcomplexexpressionsdynamicallywithouthassle.
Becausewewereabletobreakourproblemintosmallerfunctionsthatreceiveregulardata
structures,wecanuseallthetoolsavailableinElixirtoworkwithdata.Forhandlingthe
order_byparameter,itmaybebesttosimplypatternmatchonthe order_byparameter.

Forbuildingthe whereclause,wecantraversethelistofknownkeysandconvertthemto
theformatexpectedbyEcto.Forcomplexconditions,weusethe dynamicmacro.
Testingalsobecomessimpleraswecantesteachfunctioninisolation,evenwhenusing
dynamicqueries:
test"filterpublishedatbasedonthegivendate"do
assertinspect(filter_published_at("2010-04-17"))==
"dynamic([p],p.published_at>^\"2010-04-17\")"
assertinspect(filter_published_at(nil))==
"true"
end

Whileattheendofthedaysomedevelopersmayfeelmorecomfortablewithusingthe
Enum.reduce/3approach,Ecto2.0andlatergivesustheoptiontochoosewhichapproach

worksbest.
ThankstoMichałMuskałaforsuggestionsandfeedbackonthischapter.

24


4.Dynamicqueries

25