Lecture Undergraduate econometrics - Chapter 7: The multiple regression model

Chapter 7
The Multiple Regression Model
• When we turn an economic model with more than one explanatory variable into its
corresponding statistical model, we refer to it as a multiple regression model.
• Most of the results we developed for the simple regression model in Chapters 3–6 can
be extended naturally to this general case.

There are slight changes in the

interpretation of the β parameters, the degrees of freedom for the t-distribution will
change, and we will need to modify the assumption concerning the characteristics of
the explanatory (x) variables.

Slide 7.1
Undergraduate Econometrics, 2nd Edition-Chapter 7


7.1

Model Specification and the Data

When we turn an economic model with more than one explanatory variable into its
corresponding econometric model, we refer to it as a multiple regression model. Most of
the results we developed for the simple regression model in Chapter 3–6 can be extended
naturally to this general case. There are slight changes in the interpretation of the β
parameters, the degrees of freedom for the t-distribution will change, and we will need to
modify the assumption concerning the characteristics of the explanatory (x) variables.
As an example for introducing and analyzing the multiple regression model,
consider a model used to explain total revenue for a fast-food hamburger chain in the San
Francisco Bay area. We begin with an outline of this model and the questions that we
hope it will answer.

Slide 7.2
Undergraduate Econometrics, 2nd Edition-Chapter 7


7.1.1 The Economic Model
• Each week the management of a Bay Area Rapid Food hamburger chain must decide
how much money should be spent on advertising their products, and what specials
(lower prices) should be introduced for that week.
• How does total revenue change as the level of advertising expenditure changes? Does
an increase in advertising expenditure lead to an increase in total revenue? If so, is the
increase in total revenue sufficient to justify the increased advertising expenditure?
• Management is also interested in pricing strategy. Will reducing prices lead to an
increase or decrease in total revenue? If a reduction in price leads only to a small
increase in the quantity sold, total revenue will fall (demand is price inelastic); a price
reduction that leads to a large increase in quantity sold will produce an increase in
total revenue (demand is price elastic). This economic information is essential for
effective management.

Slide 7.3
Undergraduate Econometrics, 2nd Edition-Chapter 7


• We initially hypothesize that total revenue, tr, is linearly related to price, p, and
advertising expenditure, a. Thus the economic model is:
tr = β1 + β2p + β3a

(7.1.1)

where tr represents total revenue for a given week, p represents price in that week and

a is the level of advertising expenditure during that week. Both tr and a are measured
in terms of thousands of dollars.
• Let us assume that management has constructed a single weekly price series, p,
measured in dollars and cents, that describes overall prices.
• The remaining items in Equation (7.1.1) are the unknown parameters β1, β2 and β3 that
describe the dependence of revenue (tr) on price (p) and advertising (a).
• In the multiple regression model the intercept parameter, β1, is the value of the
dependent variable when each of the independent, explanatory variables takes the
Slide 7.4
Undergraduate Econometrics, 2nd Edition-Chapter 7


value zero. In many cases this parameter has no clear economic interpretation, but it is
almost always included in the regression model. It helps in the overall estimation of
the model and in prediction.
• The other parameters in the model measure the change in the value of the dependent
variable given a unit change in an explanatory variable, all other variables held
constant. For example, in Equation (7.1.1),
β2 = the change in tr ($1000) when p is increased by one unit ($1), and a is held
constant, or

β2 =

∆tr
∂tr
=
∆p ( a held constant) ∂p

Slide 7.5
Undergraduate Econometrics, 2nd Edition-Chapter 7

• The symbol ∂ stands for “partial differentiation.” It means that we calculate the
change in one variable, tr, when the variable p changes, all other factors, a, held
constant.
• The sign of β2 could be positive or negative. If an increase in price leads to an
increase in revenue, then β2 > 0, and the demand for the chain’s products is price
inelastic. Conversely, a price elastic demand exists if an increase in price leads to a
decline in revenue, in which case β2 < 0. Thus, knowledge of the sign of β2 provides
information on the price elasticity of demand. The magnitude of β2 measures the
amount of the change in revenue for a given price change.
• The parameter β3 describes the response of revenue to a change in the level of
advertising expenditure. That is,
β3 = the change in tr ($1000) when a is increased by one unit ($1000), and p is
held constant, or
Slide 7.6
Undergraduate Econometrics, 2nd Edition-Chapter 7


β3 =

∆tr
∂tr
=
∆a ( p held constant) ∂a

• We expect the sign of β3 to be positive. That is, we expect that an increase in
advertising expenditure, unless the ad is offensive, will lead to an increase in total
revenue.
• The next step along the road to learning about β1, β2 and β3 is to convert the economic

model into an econometric model.

Slide 7.7
Undergraduate Econometrics, 2nd Edition-Chapter 7


7.1.2 The Econometric Model
• The economic model in Equation (7.1.1) describes the expected behavior of many
individual franchises. As such we should write it as E(tr) = β1 + β2p + β3a, where
E(tr) is the “expected value” of total revenue.
• Weekly data for total revenue, price and advertising will not follow an exact linear
relationship. The Equation (7.1.1) describes, not a line as in Chapters 3-6, but a plane.
• The plane intersects the vertical axis at β1. The parameters β2 and β3 measure the
slope of the plane in the directions of the “price axis” and the “advertising axis,”
respectively.
• Table 7.1 shows representative weekly observations on total revenue, price and
advertising expenditure for a hamburger franchise. If we plot the data we obtain
Figure 7.1. These data do not fall exactly on a plane, but instead resemble a “cloud.”
• To allow for a difference between observable total revenue and the expected value of
total revenue we add a random error term, e = tr − E(tr). This random error represents
Slide 7.8
Undergraduate Econometrics, 2nd Edition-Chapter 7


all the factors that cause weekly total revenue to differ from its expected value. These
factors might include the weather, the behavior of competitors, a new Surgeon
General’s report on the deadly effects of fat intake, etc.
• Denoting the t’th weekly observation by the subscript t, we have
trt = E(tr) + et = β1 + β2pt + β3at + et

(7.1.2)

• The economic model in Equation (7.1.1) describes the average, systematic relationship
between the variables tr, p, and a. The expected value E(tr) is the nonrandom,
systematic component, to which we add the random error e to determine tr. Thus, tr is
a random variable. We do not know what the value of weekly total revenue will be
until we observe it.
• The introduction of the error term, and assumptions about its probability distribution,
turn the economic model into the econometric model in Equation (7.1.2).

The
Slide 7.9

Undergraduate Econometrics, 2nd Edition-Chapter 7


econometric model provides a more realistic description of the relationship between
the variables, as well as a framework for developing and assessing estimators of the
unknown parameters.
7.1.2a The General Model
• In a general multiple regression model a dependent variable yt is related to a number of
explanatory variables xt2, xt3,…, xtK through a linear equation that can be written as
yt = β1 + β2xt2 + β3xt3 +…+ βK xtK

(7.1.3)

• The coefficients β1, β2,…, βK are unknown parameters. The parameter βK measures
the effect of a change in the variable xtK upon the expected value of yt, E(yt), all other
variables held constant. The parameter β1 is the intercept term. The “variable” to
which β1 is attached is xt1 = 1.

Slide 7.10
Undergraduate Econometrics, 2nd Edition-Chapter 7


• The equation for total revenue can be viewed as a special case of Equation (7.1.3)
where K = 3, yt = trt, xt1 = 1, xt2 = pt and xt3 = at. Thus we rewrite Equation (7.1.2) as
yt = β1 + β2xt2 + β3xt3 + et

(7.1.4)

7.1.2b The Assumptions of the Model
To make the statistical model in Equation (7.1.4) complete, assumptions about the
probability distribution of the random errors, et, need to be made. The assumptions that
we introduce for et are similar to those introduced for the simple regression model in
Chapter 3. They are
1. E[et] = 0. Each random error has a probability distribution with zero mean. Some
errors will be positive, some will be negative; over a large number of observations they
will average out to zero. With this assumption we asset that the average of all the
Slide 7.11
Undergraduate Econometrics, 2nd Edition-Chapter 7


omitted variables, and any other errors made when specifying the model, is zero. Thus,
we are asserting that our model is on average correct.
2. var(et) = σ2. Each random error has a probability distribution with variance σ2. The
variance σ2 is an unknown parameter and it measures the uncertainty in the statistical
model. It is the same for each observation, so that for no observations will the model
uncertainty be more, or less, nor is it directly related to any economic variable. Errors
with this property are said to be homoskedastic.

3. cov(et, es) = 0. The covariance between the two random errors corresponding to any
two different observations is zero. The size of an error for one observation has no
bearing on the likely size of an error for another observation. Thus, any pair of errors
is uncorrelated.
4. We will sometimes further assume that the random errors et have normal probability
distributions. That is, et ~ N(0, σ2).

Slide 7.12
Undergraduate Econometrics, 2nd Edition-Chapter 7


Because each observation on the dependent variable yt depends on the random error term
et, each yt is also a random variable. The statistical properties of yt follow from those of
et. These properties are
1. E(yt) = β1 + β2xt2 + β3xt3. The expected (average) value of yt depends on the values of
the explanatory variables and the unknown parameters. This assumption is equivalent
to E(et) = 0. It says that the average value of yt changes for each observation and is
given by the regression function E(yt) = β1 + β2xt2 + β3xt3.
2. var(yt) = var(et) = σ2. The variance of the probability distribution of yt does not change
with each observation. Some observations on yt are not more likely to be further from
the regression function than others.
3. cov(yt, ys) = cov(et, es) = 0. Any two observations on the dependent variable are
uncorrelated.

Slide 7.13
Undergraduate Econometrics, 2nd Edition-Chapter 7


4. We sometimes will assume the values of yt are normally distributed about their mean.
That is, yt ~ N[(β1 + β2xt2 + β3xt3), σ2], which is equivalent to assuming that et ~ N(0,

σ2).
• In addition to the above assumptions about the error term (and hence about the
dependent variable), we make two assumptions about the explanatory variables.
• The first is that the explanatory variables are not random variables. Thus, we are
assuming that the values of the explanatory variables are known to us prior to
observing the values of the dependent variable. This assumption is realistic for our
hamburger chain where a decision about prices and advertising is made each week and
values for these variables are set accordingly. For cases in which this assumption is
untenable, our analysis will be conditional upon the values of the explanatory
variables in our sample, or, further assumptions must be made.

Slide 7.14
Undergraduate Econometrics, 2nd Edition-Chapter 7


• The second assumption is that any one of the explanatory variables is not an exact
linear function of any of the others. This assumption is equivalent to assuming that no
variable is redundant. As we will see, if this assumption is violated, a condition called
“exact multicollinearity,” the least squares procedure fails.
• To summarize then, let us construct a list of the assumptions for the general multiple
regression model in Equation (7.1.3), much as we have done in the earlier chapters, to
which we can refer as needed:

Assumptions of the Multiple Regression Model
MR1. yt = β1 + β2xt2 + … + βKxtK + et, t = 1,…,T
MR2. E(yt) = β1 + β2xt2 + … + βKxtK ⇔ E(et) = 0
MR3. var(yt) = var(et) = σ2
MR4. cov(yt, ys) = cov(et, es) = 0
Slide 7.15
Undergraduate Econometrics, 2nd Edition-Chapter 7

MR5. The values of xtK are not random and are not exact linear functions of the
other explanatory variables.
MR6. yt ~ N[(β1 + β2xt2 + β3xt3), σ2] ⇔ et ~ N(0, σ2)

Slide 7.16
Undergraduate Econometrics, 2nd Edition-Chapter 7


7.2

Estimating the Parameters of the Multiple Regression Model

We consider the problem of using the least squares principle to estimate the unknown
parameters of the multiple regression model. We will discuss estimation in the context of
the model in Equation (7.1.4), which is
yt = β1 + β2xt2 + β3xt3 + et

(7.2.1)

7.2.1 Least Squares Estimation Procedure
• With the least squares principle we minimize the sum of squared differences between
the observed values of yt and its expected value E[yt] = β1 + β2xt2 + β3xt3.
• Mathematically, we minimize the sum of squares function S(β1, β2, β3), which is a
function of the unknown parameters, given the data,
Slide 7.17
Undergraduate Econometrics, 2nd Edition-Chapter 7

T

S (β1 , β2 , β3 ) = ∑ ( yt − E[ yt ]) 2
t =1
T

= ∑ ( yt − β1 − β2 xt 2 − β3 xt 3 ) 2

(7.2.2)

t =1

• Given the sample observations yt, minimizing the sum of squares function is a
straightforward exercise in calculus. The solutions are the least squares estimates b1,
b2, and b3.
• In order to give expressions for the least squares estimates it is convenient to express
each of the variables as deviations from their means. That is, let
yt* = yt − y ,

xt*2 = xt 2 − x2 , xt*3 = xt 3 − x3

Slide 7.18
Undergraduate Econometrics, 2nd Edition-Chapter 7


• The least squares estimates b1, b2 and b3 are:

b1 = y − b2 x2 − b3 x3

b2

b3

( ∑ y x )( ∑ x ) − ( ∑ y x )( ∑ x
=
( ∑ x )( ∑ x ) − ( ∑ x x )
* *
t t2

*2
t3

*2
t2

* *
t t3

*2
t3

* *
t 2 t3

* *
t2 t3

*2
t2

*2
t2

* *
t t2

*2
t3

(7.2.3)

* *
t3 t2

* *
t2 t3

)

2

y x )( ∑ x ) − ( ∑ y x )( ∑ x
(
∑
=
( ∑ x )( ∑ x ) − ( ∑ x x )
* *
t t3

x

x

)

2

Proof:
Equation (7.2.2) can be rewritten as follows:

Slide 7.19
Undergraduate Econometrics, 2nd Edition-Chapter 7


T

T

t =1

t =1

S (b1, b2 , b3) = ∑ ( yt − yˆt )2 = ∑ ( yt − b1 − b2 xt 2 − b3 xt3)2

∂S = T ( y − b − b x − b x )(−1) = 0
t
1
2 t2
3 t3
∂b1 ∑

t =1
T

⇒ ∑ ( yt − b1 − b2 xt 2 − b3 xt 3) = 0
t =1
T

T

T

t =1

t =1

t =1

⇒ ∑ yt − Tb1 − b2 ∑ xt 2 − b3 ∑ xt3 = 0
T

T

T

t =1

t =1

t =1

⇒ Tb1 + b2 ∑ xt 2 + b3 ∑ xt3 = ∑ yt (Both sides are divided by T )
⇒ b1 + b2 x2 + b3 x3 = y
⇒ b1 = y − b2 x2 − b3 x3

Slide 7.20
Undergraduate Econometrics, 2nd Edition-Chapter 7


∂S = T ( y − b − b x − b x )(− x ) = 0
t
1
2 t2
3 t3
t2
∂b2 ∑
t =1
T

⇒ −∑ ( xt 2 yt − b1xt 2 − b2 xt22 − b3 xt 2 xt 3) = 0
t =1

T

T

T

t =1

t =1

t =1

⇒ ∑ xt 2 yt − b1∑ xt 2 − b2 ∑

xt22 − b3

T

xt 2 xt3 = 0
∑
t =1

T

T

T

T

t =1

t =1

t =1

t =1

⇒ ( y − b2 x2 − b3 x3)(∑ xt 2 ) + b2 ∑ xt22 + b3 ∑ xt 2 xt3 = ∑ xt 2 yt

T

T

T

T

t =1

t =1

t =1

t =1

T

T

T

T

T

T

t =1

t =1

t =1

t =1

t =1

t =1

⇒ y∑ xt 2 − b2 x2 ∑ xt 2 − b3 x3 ∑ xt 2 + b2 ∑

xt22 + b3

T

T

xt 2 xt 3 = ∑ xt 2 yt
∑
t =1
t =1

⇒ b2 (∑ xt22 − x2 ∑ xt 2 ) + b3(∑ xt 2 xt 3 − x3 ∑ xt 2 ) = ∑ xt 2 yt − y∑ xt 2
T

⇒ b2 ∑

t =1

( xt 2 −x2 )2 + b3

T

T

(xt 2 − x2 )( xt3 − x3) = ∑ ( xt 2 − x2 )( yt − y)
∑
t =1
t =1
Slide 7.21

Undergraduate Econometrics, 2nd Edition-Chapter 7


∂S = T ( y − b − b x − b x )(− x ) = 0
t
1
2 t2
3 t3
t3
∂b3 ∑
t =1
T

⇒ −∑ ( xt 3 yt − b1xt3 − b2 xt 2 xt3 − b3 xt23) = 0
t =1

T

T

T

T

t =1

t =1

t =1

t =1

⇒ ∑ xt 3 yt − b1∑ xt3 − b2 ∑ xt 2 xt 3 − b3 ∑ xt23 = 0
T

T

T

T

t =1

t =1

t =1

t =1

⇒ ( y − b2 x2 − b3 x3)(∑ xt3) + b2 ∑ xt 2 xt 3 + b3 ∑ xt23 = ∑ xt3 yt
T

T

T

T

T

t =1

t =1

t =1

t =1

t =1

⇒ y∑ xt3 − b2 x2 ∑ xt3 − b3 x3 ∑ xt 3 + b2 ∑ xt 2 xt3 + b3 ∑

xt23 =

T

xt3 yt
∑

t =1

T

T

T

T

T

T

t =1

t =1

t =1

t =1

t =1

t =1

⇒ b2 (∑ xt 2 xt3 − x2 ∑ xt3) + b3(∑ xt23 − x3 ∑ xt 3) = ∑ xt3 yt − y∑ xt3
T

T

t =1

t =1

T

⇒ b2 ∑ ( xt 2 −x2 )( xt 3 − x3) + b3 ∑ ( xt3 − x3 = ∑ ( xt3 − x3)( yt − y)
)2

t =1

Slide 7.22
Undergraduate Econometrics, 2nd Edition-Chapter 7


T

T

( xt 2 − x2 )( yt − y) ∑ ( xt 2 − x2 )( xt3 − x3)
∑
t =1
t =1
T

T

( xt3 − x3)( yt − y)
∑

t =1

b2 =

T

( xt3 − x3)2
∑
t =1
T

∑ ( xt 2 − x2 )2

( xt 2 − x2 )( xt3 − x3)
∑
t =1

t =1

T

T

( xt 2 − x2 )( xt3 − x3)
∑
t =1
T

( xt3 − x3)2
∑

t =1
T

T

T

∑ ( xt 2 − x2 )( yt − y)∑ ( xt3 − x3)2 − ∑ ( xt3 − x3)( yt − y)∑ ( xt 2 − x2 )( xt3 − x3)

= t =1



=

t =1

T

t =1

t =1

T

T

( xt 2 − x2 )2 ∑ ( xt3 − x3)2 − (∑ ( xt 2 − x2 )( xt3 − x3))2
∑
t =1

t =1
t =1

∑ yt*xt*2  ∑ xt*23  −  ∑ yt*xt*3  ∑ xt*2 xt*3 






∑

xt*22 





∑



 
xt*2
−
3  



∑

xt*2 xt*3 




2

Slide 7.23
Undergraduate Econometrics, 2nd Edition-Chapter 7


T

T

( xt 2 − x2 )( xt3 − x3) ∑ ( xt 2 − x2 )( yt − y)
∑
t =1
t =1
T

( xt 2 − x2
∑
t =1

b3 =

T

( xt3 − x3)( yt − y)
∑
t =1

)2

T

T

∑ ( xt 2 − x2 )2

( xt 2 − x2 )( xt 3 − x3)
∑
t =1

t =1

T

T

( xt 2 − x2 )( xt 3 − x3)
∑
t =1
T

( xt 3 − x3)2
∑
t =1

T

T

T

∑ ( xt 2 − x2 )( xt3 − x3)∑ ( xt3 − x3)( yt − y) − ∑ ( xt 2 − x2 )( yt − y)∑ ( xt 2 − x2 )( xt3 − x3)

= t =1



=

t =1

t =1

T

T

t =1

T

( xt 2 − x2 )2 ∑ ( xt3 − x3)2 − (∑ ( xt 2 − x2 )( xt3 − x3))2
∑
t =1
t =1

t =1

∑ yt*xt*3   ∑ xt*22  −  ∑ yt*xt*2  ∑ xt*3xt*2 






∑

xt*22 





∑



 
xt*2
−
3  



∑

xt*2 xt*3 




2

Slide 7.24
Undergraduate Econometrics, 2nd Edition-Chapter 7


• These formulas can be used to obtain least squares estimates in the model (7.2.1),
whatever the data values are. Looked at as a general way to use sample data, the
formulas in (7.2.3) are referred to as estimation rules or procedures and are called the
least squares estimators of the unknown parameters.
• Since their values are not known until the data are observed and the estimates
calculated, the least squares estimators are random variables.
• When applied to a specific sample of data, the rules produce the least squares
estimates, which are numeric values.

7.2.2 Least Squares Estimates Using Hamburger Chain Data
Table 7.2 contains the output obtained when the EViews computer software is used to
estimate β1, β2, and β3 for the hamburger revenue equation. For the moment, we are
concerned only with the least squares estimates, which, from the equation, are:
Slide 7.25
Undergraduate Econometrics, 2nd Edition-Chapter 7