Data Mining and Knowledge Discovery Handbook, 2 Edition part 38 pptx

350 Jean-Francois Boulicaut and Baptiste Jeudy
describe a mining algorithm but rather a pruning technique for non anti-monotonic
and non monotonic constraints. Considering a sub-lattice
˚
A of 2
I
, the problem is to
decide whether this sub-lattice can be pruned. A sub-lattice is characterized by its
maximal element M and its minimal element m, i.e., the sub-lattice is the collection
of all itemsets S such that m ⊆ S ⊆ M. To prune this sub-lattice, one must prove that
none of its elements can satisfy the constraint C . To check this, the authors introduce
the concept of negative witness: a negative witness for C in the sub-lattice
˚
A is an
itemset W such that ¬C (W ) ⇒∀X ∈
˚
A, ¬C (X). Therefore, if the constraint is not
satisfied by the negative witness, then the whole sub-lattice can be pruned. Finding
witnesses for anti-monotonic or monotonic constraints is easy : m is the witness for
all anti-monotonic constraints and M for all monotonic ones. The authors then show
how to compute efficiently witnesses for various tough constraints. For instance, for
AV G (S) >
σ
, a witness is the set m ∪
{
i ∈ M |i.v >
σ
}
. The authors also gives an
algorithm (linear in the size of I ) to compute a witness for the difficult constraint
(VA R (S) >

σ
) where VAR denotes the variance.
17.4.3 Ad-hoc Strategies
Apart from generic algorithms, many algorithms have been designed to cope with
specific classes of constraints. We select only two examples.
The FIC algorithm (Pei et al., 2001) does a depth-first exploration of the item-
set lattice. It is very efficient due to its clever data structure, a prefix-tree used to
store the database. This algorithm can compute the extended theory for a conjunction
C
am
∧C
m
∧C

where C

is convertible anti-monotonic or monotonic. A constraint
C

is convertible anti-monotonic if there exists an order on the items such that, if
itemsets are written using this order, every prefix of an itemset satisfying C

satisfies
C

. For instance, AVG(S) >
σ
is convertible anti-monotonic if the items i are or-
dered by decreasing value i.v. The main problem with convertible constraints is that
a conjunction of convertible constraints is generally not convertible.

Another example of an ad-hoc strategy is used in the c-Spade algorithm (Zaki,
2000). This algorithm is used to extract constrained sequences where each event
in the sequences is dated. One of the constraints, the max −gap constraint, states
that two consecutive events occurring in a pattern must not be further apart than a
given maximum gap. This constraint is neither anti-monotonic nor monotonic and a
specific algorithm has been designed for it.
17.4.4 Other Directions of Research
Among others, let us introduce here three important directions of research.
Adaptive Pruning Strategies
We mentioned the trade-off between anti-monotonic pruning which is known to be
quite efficient and pruning based on non anti-monotonic constraints. Since the se-
lectivity of the various constraints is generally unknown, a quite exciting challenge
17 Constraint-based Data Mining 351
is to look for adaptive strategies which can decide of the pruning strategy dynam-
ically. (Bonchi et al., 2003A, Bonchi et al., 2003B) propose algorithms for fre-
quent itemsets under syntactical monotonic constraints. (Albert-Lorincz and Bouli-
caut, 2003) considers frequent sequence mining under regular expression constraints.
These are promising approaches to widen the applicability of constraint-based min-
ing techniques in real contexts.
Combining Constraints and Condensed Representations
A few papers, e.g., (Boulicaut and Jeudy, 2000, Bonchi and Lucchese, 2004), deal
with the problem of extracting constrained condensed representation. In these works,
the aim is to compute a condensed representation of the extended theory
Th
x
(D,2
I
,C
am
∧C

m
,freq). In (Boulicaut and Jeudy, 2000), the authors use free
itemsets, i.e., their algorithm computes the extended theory Th
x
(D,2
I
,C
am
∧C
m
∧
C
free
,freq). In (Bonchi and Lucchese, 2004), the authors use closed itemsets, i.e.,
their algorithm computes the extended theory Th
x
(D,2
I
,C
am
∧C
m
∧C
clos
,freq).
However, in these two works, the definition of free sets and closed sets have been
modified to be able to regenerate the extended theory Th
x
(D,2
I

,C
am
∧C
m
,freq)
from the extracted theories. This kind of research combines the advantages of both
condensed representations and constrained mining which result in very efficient al-
gorithms.
Constraint-based Mining of more Complex Pattern
Domains
Most of the recent results have concerned simple local pattern discovery tasks like
the ones based on itemsets or sequences. We believe that inductive querying is much
more general. Many open problems are however to be addressed. For instance, even
constraint-based mining of association rules is already much harder than constraint-
based mining of itemsets (Lakshmanan et al., 1999,Jeudy and Boulicaut, 2002). The
recent work on the MINE RULE query language (Meo et al., 1998) is also typical
of the difficulty to optimize constraint-based association rule mining (Meo, 2003).
When considering model mining under constraints (e.g., classifier design or clus-
tering), only very preliminary approaches are available (see, e.g., (Garofalakis and
Rastogi, 2000)). We think that this will be a major issue for research in the next few
years. For instance, for clustering, it seems important to go further than the classical
similarity optimization constraints and enable to specify other constraints on clusters
(e.g., enforcing that some objects are or are not within the same clusters).
17.5 Conclusion
In this chapter, we have considered constraint-based mining approaches, i.e., the core
techniques for inductive querying.
352 Jean-Francois Boulicaut and Baptiste Jeudy
This domain has been studied a lot for simple pattern domains like itemsets or
sequences. Rather general forms of inductive queries on these domains (e.g., ar-
bitrary boolean expressions over monotonic and anti-monotonic constraints) have

been considered. Beside the many ad-hoc algorithms, an interesting effort has con-
cerned generic algorithms. Many open problems are still there: how to solve tough
constraints?, how to design relevant approximation or relaxation schemes? how to
combine constraint-based mining with condensed representations, not only for sim-
ple pattern domains but also more complex ones?
Moreover, within the inductive database framework, the problem is to optimize
sequences of queries and typically sequences of correlated inductive queries. It is
crucial to consider that the optimization of a query and thus constraint-based mining
must also take into account the previously solved queries. Looking for the formal
properties between inductive queries, especially containment, is thus a major priority.
Here again, we believe that condensed representations might play a major role.
Last but not the least, a quite challenging problem is to consider from where
the constraints come. The analysts can think in terms of constraints or declarative
specifications which are not supported by the available solvers: an obvious example
could be unexpectedness or novelty w.r.t. some explicit background knowledge. To
be able to derive appropriate inductive queries based on a limited number of primi-
tives (and some associated solvers) from the constraints expressed by the analysts is
challenging.
References
R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A. I. Verkamo. Fast discovery of
association rules. In Advances in Knowledge Discovery and Data Mining, pages 307–
328. AAAI Press, 1996.
H. Albert-Lorincz and J F. Boulicaut. Mining frequent sequential patterns under regular
expressions: a highly adaptative strategy for pushing constraints. In Proc. SIAM DM’03,
pages 316–320, 2003.
Y. Bastide, N. Pasquier, R. Taouil, G. Stumme, and L. Lakhal. Mining minimal non-
redundant association rules using frequent closed itemsets. In Proc. CL 2000, volume
1861 of LNCS, pages 972–986. Springer-Verlag, 2000.
Y. Bastide, R. Taouil, N. Pasquier, G. Stumme, and L. Lakhal. Mining frequent patterns with
counting inference. SIGKDD Explorations, 2(2):66–75, 2000.

R. J. Bayardo. Efficiently mining long patterns from databases. In Proc. ACM SIGMOD’98,
pages 85–93, 1998.
F. Bonchi, F. Giannotti, A. Mazzanti, and D. Pedreschi. Adaptive constraint pushing in fre-
quent pattern mining. In Proc. PKDD’03, volume 2838 of LNAI, pages 47–58. Springer-
Verlag, 2003A.
F. Bonchi, F. Giannotti, A. Mazzanti, and D. Pedreschi. Examiner: Optimized
level-wise frequent pattern mining with monotone constraints. In Proc.
IEEE ICDM’03, pages 11–18, 2003B.
F. Bonchi, F. Giannotti, A. Mazzanti, and D. Pedreschi. Exante: Anticipated data reduction
in constrained pattern mining. In Proc. PKDD’03, volume 2838 of LNAI, pages 59–70.
Springer-Verlag, 2003C.
17 Constraint-based Data Mining 353
F. Bonchi and C. Lucchese. On closed constrained frequent pattern mining. In Proc. IEEE
ICDM’04 (In Press), 2004.
J F. Boulicaut. Inductive databases and multiple uses of frequent itemsets: the cInQ ap-
proach. In Database Technologies for Data Mining - Discovering Knowledge with In-
ductive Queries, volume 2682 of LNCS, pages 1–23. Springer-Verlag, 2004.
J F. Boulicaut and A. Bykowski. Frequent closures as a concise representation for binary
Data Mining. In Proc. PAKDD’00, volume 1805 of LNAI, pages 62–73. Springer-Verlag,
2000.
J F. Boulicaut, A. Bykowski, and C. Rigotti. Approximation of frequency queries by mean
of free-sets. In Proc. PKDD’00, volume 1910 of LNAI, pages 75–85. Springer-Verlag,
2000.
J F. Boulicaut, A. Bykowski, and C. Rigotti. Free-sets : a condensed representation of
boolean data for the approximation of frequency queries. Data Mining and Knowledge
Discovery, 7(1):5–22, 2003.
J F. Boulicaut and B. Jeudy. Using constraint for itemset mining: should we prune or not?
In Proc. BDA’00, pages 221–237, 2000.
J F. Boulicaut and B. Jeudy. Mining free-sets under constraints. In Proc. IEEE IDEAS’01,
pages 322–329, 2001.

C. Bucila, J. E. Gehrke, D. Kifer, and W. White. Dualminer: A dual-pruning algorithm for
itemsets with constraints. Data Mining and Knowledge Discovery, 7(4):241–272, 2003.
D. Burdick, M. Calimlim, and J. Gehrke. MAFIA: A maximal frequent itemset algorithm
for transactional databases. In Proc. IEEE ICDE’01, pages 443–452, 2001.
A. Bykowski and C. Rigotti. DBC: a condensed representation of frequent patterns for
efficient mining. Information Systems, 28(8):949–977, 2003.
T. Calders and B. Goethals. Mining all non-derivable frequent itemsets. In Proc. PKDD’02,
volume 2431 of LNAI, pages 74–85. Springer-Verlag, 2002.
B. Cr
´
emilleux and J F. Boulicaut. Simplest rules characterizing classes generated by delta-
free sets. In Proc. ES 2002, pages 33–46. Springer-Verlag, 2002.
L. De Raedt. A perspective on inductive databases. SIGKDD Explorations, 4(2):69–77,
2003.
L. De Raedt, M. Jaeger, S. Lee, and H. Mannila. A theory of inductive query answering. In
Proc. IEEE ICDM’02, pages 123–130, 2002.
L. De Raedt and S. Kramer. The levelwise version space algorithm and its application to
molecular fragment finding. In Proc. IJCAI’01, pages 853–862, 2001.
M. M. Garofalakis and R. Rastogi. Scalable Data Mining with model constraints. SIGKDD
Explorations, 2(2):39–48, 2000.
M. M. Garofalakis, R. Rastogi, and K. Shim. SPIRIT: Sequential pattern mining with regular
expression constraints. In Proc. VLDB’99, pages 223–234, 1999.
B. Goethals and M. J. Zaki, editors. Proc. of the IEEE ICDM 2003 Workshop on Frequent
Itemset Mining Implementations, volume 90 of CEUR Workshop Proceedings, 2003.
D. Gunopulos, R. Khardon, H. Mannila, S. Saluja, H. Toivonen, and R. S.
Sharm. Discovering all most specific sentences. ACM Transactions on
Database Systems, 28(2):140–174, 2003.
T. Imielinski and H. Mannila. A database perspective on knowledge discovery. Communi-
cations of the ACM, 39(11):58–64, 1996.
B. Jeudy and J F. Boulicaut. Optimization of association rule mining queries. Intelligent

Data Analysis, 6(4):341–357, 2002.
D. Kifer, J. E. Gehrke, C. Bucila, and W. White. How to quickly find a witness. In Proc.
ACM PODS’03, pages 272–283, 2003.
354 Jean-Francois Boulicaut and Baptiste Jeudy
S. Kramer, L. De Raedt, and C. Helma. Molecular feature mining in HIV data. In Proc.
ACM SIGKDD’01, pages 136–143, 2001.
L. V. Lakshmanan, R. Ng, J. Han, and A. Pang. Optimization of constrained frequent set
queries with 2-variable constraints. In Proc. ACM SIGMOD’99, pages 157–168, 1999.
D I. Lin and Z. M. Kedem. Pincer search: An efficient algorithm for discovering the maxi-
mum frequent sets. IEEE Transactions on Knowledge and Data Engineering, 14(3):553–
566, 2002.
H. Mannila and H. Toivonen. Multiple uses of frequent sets and condensed representations.
In Proc. KDD’96, pages 189–194. AAAI Press, 1996.
H. Mannila and H. Toivonen. Levelwise search and borders of theories in knowledge discov-
ery. Data Mining and Knowledge Discovery, 1(3):241–258, 1997.
C. Mellish. The description identification problem. Artificial Intelligence,
52(2):151–168, 1992.
R. Meo. Optimization of a language for Data Mining. In Proc. ACM SAC’03 - Data Mining
Track, pages 437–444, 2003.
R. Meo, G. Psaila, and S. Ceri. An extension to SQL for mining association rules. Data
Mining and Knowledge Discovery, 2(2):195–224, 1998.
T. Mitchell. Generalization as search. Artificial Intelligence, 18(2):203–226, 1980.
R. Ng, L. V. Lakshmanan, J. Han, and A. Pang. Exploratory mining and
pruning optimizations of constrained associations rules. In Proc. ACM
SIGMOD’98, pages 13–24, 1998.
N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Efficient mining of association rules using
closed itemset lattices. Information Systems, 24(1):25–46, 1999.
J. Pei, G. Dong, W. Zou, and J. Han. On computing condensed frequent pattern bases. In
Proc. IEEE ICDM’02, pages 378–385, 2002.
J. Pei, J. Han, and L. V. S. Lakshmanan. Mining frequent itemsets with convertible con-

straints. In Proc. IEEE ICDE’01, pages 433–442, 2001.
R. Srikant, Q. Vu, and R. Agrawal. Mining association rules with item constraints. In Proc.
ACM SIGKDD’97, pages 67–73, 1997.
M. J. Zaki. Sequence mining in categorical domains: incorporating constraints. In Proc.
ACM CIKM’00, pages 422–429, 2000.
18
Link Analysis
Steve Donoho
Mantas, Inc.
Summary. Link analysis is a collection of techniques that operate on data that can be rep-
resented as nodes and links. This chapter surveys a variety of techniques including subgraph
matching, finding cliques and K-plexes, maximizing spread of influence, visualization, find-
ing hubs and authorities, and combining with traditional techniques (classification, clustering,
etc). It also surveys applications including social network analysis, viral marketing, Internet
search, fraud detection, and crime prevention.
Key words: Link analysis, Social network analysis, Graph theory
18.1 Introduction
The term ”link analysis” does not refer to one specific technique or algorithm. Rather
it refers to a collection of techniques that are bound together by the type of data they
operate on. Link analysis techniques are applied to data that can be represented as
nodes and links as in Figure 18.1.
A node represents an entity such as a person, a document, or a bank account.
Nodes are sometimes referred to as ”vertices.” A link represents a relationship be-
tween two entities such as a parent/child relationship between two people, a reference
relationship between two documents, or a transaction between two bank accounts.
Links are sometimes referred to as ”edges.” Because links show relationships among
entities, this type of data is often referred to as relational data.
This is as opposed to attribute vector data used by many other unsupervised and
supervised Data Mining techniques. In most standard Data Mining techniques, data
is represented as a set of tuples (a vector of attribute values). Each tuple represents

an entity, but there is no explicit data about relationships among entities. In link
analysis, information exists about the relationships among entities, and analysis of
these relationships is the focus of the field.
The roots of link analysis predate the use of modern computers. Law enforce-
ment officials have carried out manual link analysis for many years. When a crime
is investigated, a network such as in Figure 18.1 is drawn where the nodes represent
O. Maimon, L. Rokach (eds.), Data Mining and Knowledge Discovery Handbook, 2nd ed.,
DOI 10.1007/978-0-387-09823-4_18, © Springer Science+Business Media, LLC 2010
356 Steve Donoho
Fig. 18.1. Node and Link Data Used by Link Analysis Techniques.
people, weapons, crime scenes, etc. One person may be linked to another if they
are family, friends, roommates, or business partners. A person may be linked to a
weapon if it is registered in his name or if it was found at his home. Once a network
of relationships is drawn out, the bigger picture of a crime emerges from the details.
Holes in the network become apparent, and they are areas for further investigation.
Hypotheses can be formed and tested.
Sociologists also performed manual link analysis long before there were com-
puters. The structure of a clan or tribe would be mapped out with nodes represent-
ing people and links representing family, work, or social relationships. From this a
sociologist could deduce who held powerful positions within the clan, who might
influence who else, how information might spread within the clan, and what factions
might arise.
The advent of computers allowed these techniques to become much more wide-
spread and to be applied on a much larger scale. All 10 million of a bank’s customers
can be analysed for money laundering relationships. The hundreds of millions of
documents on the Internet can be analysed to determine which are most respected
and reliable. Large communities can be analysed to determine how information and
opinions spread and who are the most influential individuals.
This chapter surveys the techniques that fall under the umbrella of link analysis
and how these techniques are being applied. Section 18.2 presents some key con-

cepts from the field of Social Network Analysis. Section 18.3 examines how link
analysis techniques are used to improve search engine results. Section 18.4 looks at
recent link analysis ideas emerging from the field of viral marketing. Section 18.5
shows how fraud detection and law enforcement have presented unique challenges
and opportunities for link analysis. Finally, Section 18.6 surveys recent combinations
of link analysis with traditional Data Mining techniques.
18 Link Analysis 357
18.2 Social Network Analysis
The field of social network analysis (Wasserman, 1994,Hanneman, 2001) has devel-
oped over many years as sociologists developed formal methods of studying groups
of people and their relationships. When studying a social network, there are many
questions sociologists are interested in answering:
1. Which people are powerful?
2. Which people influence other people?
3. How does information spread within the network?
4. Who is relatively isolated, and who is well connected?
5. In a disagreement, who is likely to side with whom?
6. What roles do people play in an organization, and who has similar roles?
While concepts such as powerful, influential, isolated, and connected are somewhat
subjective, social network analysis methods give us a baseline for measuring and
making comparisons.
Fig. 18.2. Three Networks to Illustrate an Individual’s Power within a Network.
Many things can make a person powerful within a group. Consider the shaded
nodes in the three networks shown in Figure 18.2 (Hanneman, 2001). The person at
the center of the star intuitively seems more powerful than the one in the circle or the
one at the end of the line. If the people in the star want to communicate with each
other they have to go through the center person, and that person has the power to
either facilitate or hamper communication. If the people in the star want to engage
in business, they have to go through the person in the center, and that person has the
power to charge a fee as the middleman. In contrast, the shaded node in the circular

network is the most convenient path of communication or trade for some nodes, but
he is not the only path. Intuitively, he has less power than the center of the star. The
shaded node at the end of the line is dependant on others for communication and
trade but has no one who is dependant on him. Intuitively, he has little or no power.
The networks in Figure 18.2 illustrate how ”centrality” is one measure of power.
The node at the center of the star derives its power from being in the center of its
network. The shaded nodes in the circle and line are less central to their networks
358 Steve Donoho
and are therefore less powerful. Some quantitative methods of measuring centrality
are:
1. Degree. The shaded node in the star network is linked to six other nodes and thus
has a degree of six. All the other nodes in the star have a degree of one and are
comparatively less central. All the nodes in the circle have the same degree: two.
The shaded node in the line has a degree of one and is thus slightly less central
than other nodes in the line with degree two.
2. Closeness. The average distance from the shaded node in the star to all other
nodes is 1.0. This node has very direct access to everyone else. Other nodes
in the star have an average distance of 1.8. All the nodes in the circle have an
average distance of 2.0. The node at the end of the line has an average distance
of 3.5 whereas the node in the center of the line has an average distance of 2.0.
3. Betweenness. The shaded node in the star is between all other 15 pairs of nodes.
In the circle there are two paths between each pair of nodes. The shaded node in
the circle is on a path between all other 15 pairs, but since there is an alternative
path between each pair, the shaded node is on 50% of the paths between pairs.
The node at the end of the line is between no pairs. The node one from the end
of the line is on paths between 5 pairs (33% of 15 paths). The node at the center
of the line is on paths between 9 pairs (60% of 15 paths).
4. Cutpoints. Related to betweenness, cutpoints are nodes that if removed divide the
network into unconnected systems. These nodes hold particular power because
they are the only point of contact between otherwise disconnected networks. If

the center of the star is removed, six disconnected systems result. If a node in the
circle is removed, the network is still connected. If a non-end node is removed
from the line, two disconnected systems result.
A clique is a small, highly-interconnected group within a larger network. Cliques
are of interest for several reasons. Ideas or information may spread extremely quickly
within a clique because of the high connectivity. Members of a clique often act and
behave as a cohesive unit. Disputes may form between cliques (”factions”). A person
can be described with respect to the clique(s) they belong to. A person who is only
connected to people in his clique is called a ”local” and is strongly influenced by the
clique. A person who belongs to many cliques is called a ”cosmopolitan” and serves
to bring outside ideas and information into a clique.
The most strict definition of a clique is a complete subgraph (all nodes in the
clique must be linked to all other nodes). A couple more relaxed definitions are:
1. K-plexes. A group of N nodes is a K-plex if each of the nodes is connected to at
least N-K other nodes in the group. Intuitively, if K=2 then every member of the
clique has to be connected to all but two of the other members.
2. K-cores. The definition of a K-core is slightly more relaxed than that of a K-plex.
A K-core is a maximal group of nodes all of which are connected to at least K
other nodes in the group. For example, if K=4 then every member of the clique
is connected to at least 4 other clique members.
18 Link Analysis 359
The concept of ”equivalence” is very important within social networks. It makes it
possible to determine if a person is playing a particular role within a network. This
allows both intra-network comparisons (one node has the same role as another node
within one network) and inter-network comparison (two nodes in different networks
are playing the same role). Two measures of equivalence are:
1. Structural Equivalence. This is a strict measure of equivalence between two
nodes. Two nodes are exactly structurally equivalent if they are linked to exactly
the same other nodes. If not exactly equivalent, the degree of partial structural
equivalence can be measured using the degree of overlap in nodes they are linked

to.
2. Regular equivalence. Regular equivalence is a less strict definition than structural
equivalence. Two nodes have regular equivalence if the nodes they are linked to
are regular equivalents. For example, Fred Flintstone is the regular equivalent of
Barney Rubble because Fred is the husband of Wilma, and Barney is the husband
of Betty, and Wilma and Betty are regular equivalents.
On a broader scale, equivalence of nodes lays the groundwork for measuring the
similarity of one whole social network to another whole social network. This is useful
for matching a network against a known template in order to identify the nature of
the network as will be seen in Section 5 on Fraud Detection and Law Enforcement.
Many groups such as academic circles, fraud rings, business circles, shoppers
with common interests, and professional societies can be represented as social net-
works. Because of this, Social Network Analysis lays the groundwork for many im-
portant real-world applications.
18.3 Search Engines
The Internet is rich in relational data by the simple fact that web pages are linked to
other web pages. While traditional search techniques such as keyword searches fo-
cus exclusively on the content of a single page, newer techniques (Page et al., 1998,
Kleinberg, 1999) exploit relationships among pages. A user performing a search
wants to find results that are not only relevant but are also authoritative and reli-
able. A keyword search on ”stock market” will not only return authoritative sites
such as the NASDAQ and NYSE pages, it will also return pages from thousands of
self-proclaimed gurus selling books, software, and advice. The truly reliable sources
of information are likely to be lost among the self-proclaimed gurus. Is there a way
to separate the wheat from the chaff? This is where relational information contained
in links comes into play.
An authoritative site such as the NASDAQ is likely to be recognized as authori-
tative by many people; therefore, many other sites are likely to point to the NASDAQ
site. But a self-proclaimed stock market guru is less likely to have many other sites
pointing to his site unless there truly is some merit to what he has to say. When one

site references another site, it is in fact declaring that that site has some merit – it
is casting a vote for the value and importance of the other site. Conceptually, this