Discrrete mathematics for computer science statistics

Probability and Statistics


Probability vs. Statistics
• In probability, we build up from the
mathematics of permutations and
combinations and set theory a
mathematical theory of how outcomes of an
experiment will be distributed
• In statistics we go in the opposite direction:
We start from actual data, and measure it to
determine what mathematical model it fits
• Statistical measures are estimates of
underlying random variables


Basic Statistics
• Let X be a finite sequence of numbers
(data values) x1, …, xn
• E.g. X = 1, 3, 2, 4, 1, 4, 1 (n=7)
• Order doesn’t matter but we need a way
of allowing duplicates (“multiset”)
• Some measures:
– Maximum: 4
– Minimum: 1
– Median (as many ≥ as ≤):
• 1, 1, 1, 2, 3, 4, 4 so median = 2

– Mode (maximum frequency): 1

Sample Mean
• Let X be a finite sequence of numbers x1, …,
xn.
• The sample mean of
1 Xn is what we usually call
xi
the average:  X =

∑
n
i=1

• For example, X = 1, 3, 2, 4, 1, 4, 1 , then
μX=16/7
• Note that the mean need not be one of the
data values.
• We might as well write this as E[X] following
the notation used for random variables


Sample Variance
• The sample variance of a sequence
of data points is the mean of the
square of the difference from the
sample mean: 


Var(X)        
 

Standard Deviation
• Standard deviation is the square root
of the variance:
n

1
2
 = ∑ (xi − μ X )
n i=1
2
X

• σ is a measure of spread in the same
units as the data


σ Measures “Spread”
•
•
•
•
•
•
•

X = 1, 2, 3
μ=2
σ2 = (1/3) ∙ ((1-2)2+(2-2)2+(3-2)2) = 2/3
σ ≈ .82

Y = 1, 2, 3, 4, 5
μ=3
σ2 = (1/5) ∙ ((1-3)2+(2-3)2+(3-3)2+(4-3)2+(53)2)
= 10/5
σ ≈ 1.4


Small σ Indicates “Centeredness”
•
•
•
•
•
•
•
•
•
•
•

X = 1, 2, 3
σ ≈ .82
Z = 1, 2, 2, 2, 3
μ=2
σ2 = (1/5) ∙ ((1-2)2+3∙(2-2)2+(3-2)2) = 2/5
σ ≈ .63
W = 1, 1, 2, 3, 3
μ=2
σ2 = (1/5) ∙ (2∙(1-2)2+(2-2)2+2∙(3-2)2) = 4/5
σ ≈ .89

(“Bimodal”)


Covariance
• Sometimes two quantities tend to
vary in the same way, even though
neither is exactly a function of the
other
• For example,Weight
height and weight of
people
Height


Covariance for Random Variables
• Roll two dice. Let X = larger of the two values,
Y = sum of the two values
• Mean of X = (1/36) ×
(1×1 [only possibility is (1,1)]
+3×2 [(1,2), (2,1), (2,2)]
+5×3 [(1,3), (2,3), (3,3), (3,2), (3,1)]
+7×4 + 9×5 + 11×6)
= 4.47
Mean of Y = 7
How do we say that X tends to be large when Y
is large and vice versa?


Joint Probability
• f(x,y) = Pr(X=x and Y=y) is a

probability
• Sums to 1 over all possible x and y
• Pr(X=1 and Y=12) = 0
• Pr(X=5 and Y=9) = 2/36
• Pr(X≤5 and Y≥8) = 4/36
[(4,4), (4,5), (5,4), (5,5)]


Covariance of Random
Variables
• Cov(X, Y) = E[ (X − μX) ∙ (Y −μY) ]
• INSIDE the brackets, each of X and Y is
compared to its own mean
• The OUTER expectation is with respect to the
joint probability that X=x AND Y=y
• Positive if X tends to be greater than its mean
when Y is greater than its mean
• Negative if X tends to be greater than its
mean when Y is less than its mean
• But what are the units?


Sample Covariance
• Suppose we just have the data x1, …,
xN and y1, …, yN and we want to know
the extent to which these two sets of
values covary (eg height and
weight). The sample covariance is


1
(xi        

n 

• An estimate of the covariance


A Better Measure:
Correlation

• Correlation is Covariance scaled to [-1,1]

 XY

Cov(X,Y )
E[(X − μ X )⋅(Y − μ Y )]
=
=
σ X ⋅σ Y
Var(X)Var(Y )

• This is a unitless number!
• If X and Y vary in the same direction then
correlation is close to +1
• If they vary inversely then correlation is
close to -1
• If neither depends on the mean of the
other than the correlation is close to 0

Positively Correlated Data


Correlation Examples

/>

FINIS