08 link prediction (sbm)

CS224W: Social and Information Network Analysis
Jure Leskovec, Stanford University




[LibenNowell-Kleinberg ‘03]

¡

The link prediction task:
§ Given ![#$ , #$& ] a graph on edges
up to time #$& ,output a ranked list L
of links (not in ![#$ , #$& ]) that are
predicted to appear in ![#( , #(& ]

¡

Evaluation:

![#$ , #$& ]
![#( , #(& ]

§ n = |Enew|: # new edges that appear during
the test period [#( , #(& ]
§ Take top n elements of L and count correct edges
10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

2

[LibenNowell-Kleinberg ‘03]

¡

Predict links in a evolving collaboration
network

¡

Core: Because network data is very sparse
§ Consider only nodes with degree of at least 3
§ Because we don't know enough about nodes with less than
3 edges to make good inferences

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

3


¡

Methodology:
§ For each pair of nodes (x,y) compute score c(x,y)
§ For example, c(x,y) could be the # of common neighbors
of x and y

§ Sort pairs (x,y) by the decreasing score c(x,y)
§ Note: Only consider/predict edges where
both endpoints are in the core (deg. ≥ 3)

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

X

X

§ Predict top n pairs as new links
§ See which of these links actually
appear in ![#$ , #$& ]

4


[LibenNowell-Kleinberg ‘03]

¡

Different scoring functions !(#, %) =
§
§
§
§
§
§

Graph distance: (negated) Shortest path length
Common neighbors: |Γ * ∩ Γ(,)|
Jaccard’s coefficient: Γ * ∩ Γ , /|Γ * ∪ Γ(,)|
Adamic/Adar: ∑0∈2 3 ∩2(4) 1/log |Γ(9)|
Γ(x) … neighbors
Preferential attachment: |Γ * | ⋅ |Γ(,)|
of node x
PageRank: ;3 (,) + ;4 (*)

§ ;3 , … stationary distribution score of y under the random walk:
§ with prob. 0.15, jump to x
§ with prob. 0.85, go to random neighbor of current node

¡

Then, for a particular choice of c(·)

§ For every pair of nodes (x,y) compute c(x,y)
§ Sort pairs (x,y) by the decreasing score c(x,y)
§ Predict top n pairs as new links

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

5


[LibenNowell-Kleinberg ’ 03]

Performance score: Fraction
of new edges that are guessed
correctly.

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

6



¡

Link prediction
§ Local structure: Link prediction via proximity
§ Global structure: Stochastic Blockmodels
§ Another way to predict links is to identify communities.
§ We can then calculate link probabilities within and
between communities.

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

8


¡

¡

We often think of networks being organized into
modules, cluster, communities.
Blockmodels:
§ Divide the nodes of the network into distinct sets, or
"blocks", where all nodes in the same block have the same
pattern of connection to nodes in other blocks.

J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,

9


J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,

10


¡

Stochastic Block Model is a generative model for blocks,
groups, or communities in networks.

¡

There are ! vertices which belong to " groups
§ Each vertex i belongs to a group #$ ∈ {1, … , "};

¡

Given +, the edges are generated
independently,
§ Each pair of vertices is connected
with a probability that depends only
on their groups:
§ Pr[ /, 0 ∈ 1] = 456 57 .
Adjacency matrix of a SBM. The probability of
edges are different within and between blocks.

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

11


¡

Network of 60 nodes and 6 communities generated
according to a stochastic blockmodel.

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

12


¡

In the symmetric case (SSBM):
§ ci are chosen a priori independently and uniformly.
§ Pr[(i, j) ∈ E] depends only on whether i and j are in
the same or different groups,
*+, .
§ Pr $, & ∈ ' = )
*234 .

if 0+ = 01
if 0+ ≠ 01

§ We often assume that 6in > 6out, i.e., that vertices are
more likely to connect to others in the same group.
§ The case 6in = 6out is the classic Erdős-Rényi random
graph.
10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

13


[Faskowitz et al. 2018]

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

14

-~SBM(3, 5, +)

¡

§ 3 = 30, 5 = 3
§ c = [0.3, 0.2, 0.5]

0.3 0.1 0.1
+= 0.1 0.2 0.1
0.1 0.1 0.2

!" = 0.3

!) = 0.5
!' = 0.2

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

15


¡

'~SBM(-, /, !)
§ - = 30, / = 3
§ c = [0.3, 0.2, 0.5]

42 = 0.3
122 = 0.3

0.3 0.1 0.1
!= 0.1 0.2 0.1
0.1 0.1 0.2

126 = 0.1

125 = 0.1

156 = 0.1

46 = 0.5
155 = 0.2

45 = 0.2
155 = 0.2
10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

16


¡

Some background:
§ Bayes rule: ! " # =

%('|))%())
%(')

§ Prior probability: our belief before observation.
§ Posterior probability: conditional probability after
observation.
§ Bayes rule can be used to update a prior belief
based on some observation.
Observation
Posterior probability

Prior probability

!(#|")+(,)
+ ,- =
!(#)

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

17


¡

More background:
§ Bayes rule:

Likelihood
Observation
Prior probability
Posterior probability

%('|))!(")
! "# =
%(')
§ Conjugate prior: if prior and posterior probability
distributions are in the same family, prior distribution is
conjugate for the likelihood function.
§ Using conjugate priors can simplify the inference.
Likelihood

Conjugate prior

Bernoulli

Beta

Categorical

Dirichlet

+ ,; . = ./ (1 − .)23/ , , ∈ {0,1}

+ 9 = B C = .D ,

C = {.2 , … , ./ }

+ 9; :, β = cnt. 9 @32 (1 − 9)A32

+ 92 , … , 9/ ; :2 , … , :/

H

1
@ 32
=
F 9D I
'(:)
DG2

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

18


¡

Conjugate prior: What does this mean?
§ !" , !$ = shape parameters ~ number of nodes in &' , &(
§ )*+ *, ~ Beta(!" , !$ )

Prior distribution

§ 345 ~ Bernoulli()*+ *, )

Likelihood

§ Posterior distribution:
§ )*+ *, |=45 ~ Beta !" + =45 , !$ + ?45 − =45
§ =45 = number of observed
edges between H4 and H5 .

!" , !$ = 0.5
!" = 2, !$ = 8
!" = 5, !$ = 5
!" = 8, !$ = 1
!" = 1, !$ = 1

§ ?45 = number of possible edges
between categories H4 and H5 .

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

19


¡

Conjugate prior: What does this mean?
§ ! = ($% , … , $( ) = concentration hyperparameter
§ *|! = ,% , … , ,( ~ Dirichlet(!)
Prior distribution
§ 6|* = 7% , … , 7( ~ Categorical(*)

Likelihood
§ Posterior distribution:
§ <= = number of occurrences of category >.
§ *|@, ! ~ Dirichlet(A + !) = Dirichlet <% + $% , … , <( + $(
One class has a
higher probability

Uniform
distribution

Equal
probabilities

Class 3 has a
higher probability

Sample 1
Sample 2
…

Sample 10
10/18/18

Dirichlet(.1, . 1, . 1)

Dirichlet(1,1,1)

Dirichlet(10,10,10) Dirichlet(2,5,15)

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

[Stringham]
20


¡

Bayesian version of the !"#(%, ', ():
§ Probabilities of communities
§ * ~ Dirichlet 45, … , 47 ,

§ Assignment of nodes to communities
§ 89 ~ Categorical * ,

> = 1, … , A

§ Edge probabilities within/between communities
§ BCD CE ~ Beta G5, GH ,

89 , 8I = 1, … , J

§ If G5 = GH = 1 ⟹ BCD CE ~L(0,1).

Đ Adjacency matrix
Đ N9I ~ Bernoulli(BCD CE ),
10/18/18

N {0,1}TìT

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

21


¡

Given any set of values for !, #, we can generate
random networks using the Bayesian SBM model.
§ Example:
§ $ =25
§ & =2

2 = 3, 5
- = [0.4, 0.6]

6 2
2 3
4 8
40 =
8 7
4. =

'=

0.3
0.1

0.1
0.2

-. = 0.4
'.. = 0.3
'.0 = 0.1
-0 = 0.6
'00 = 0.2
10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

22


¡

Given any set of values for !, #, we can generate
random networks using the Bayesian SBM model.
.(

.)

…

.,

'((

.(
.)

'))

$=

'*)

')*

✓
⟹

'**

&=

'++
',,

.,

Latent structure of
the stochastic blockmodel
10/18/18

Adjacency matrix of a graph generated
with the stochastic blockmodel

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

23

How can we infer the latent structure
(!, #, $) for an observed network?
!(

!(

!)

…

!,

$((

!)

$))
$*)

%=

$)*

?

⟹

¡

$**

'=

$++
$,,

!,

Latent structure of
the stochastic blockmodel
10/18/18

Adjacency matrix of
an observed network

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

24


¡

How can we infer the latent structure (!, #, $) for an
observed network?
§ Bayes rule!
Observation

Prior probability

Posterior probability

.(0|%)*(+)
* +, =
.(0)

§ Observations: adjacency matrix %.
§ Latent variables: !, #, $
§ &, ': community detection
§ ): link prediction

10/18/18

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis,

25