DSpace at VNU: Parallel combinatorial algorithms for multi-sets and their applications

International Journal of Software Engineering
and Knowledge Engineering
Vol. 23, No. 1 (2013) 81À99
#
.c World Scienti¯c Publishing Company
DOI: 10.1142/S0218194013400068

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

PARALLEL COMBINATORIAL ALGORITHMS
FOR MULTI-SETS AND THEIR APPLICATIONS

HOANG CHI THANH
Department of Computer Science
VNU University of Science
334 - Nguyen Trai, Hanoi, Vietnam


In this paper we extend some well-known notions of combinatorics on multi-sets such as
iterative permutation, multi-subset, iterative combination and then construct new e±cient
algorithms for generating all iterative permutations, multi-subsets and iterative combinations
of a multi-set. Applying the parallelizing method based on output decomposition we parallelize
the algorithms. Furthermore, we use these algorithms to solve an optimal problem of work
arrangement and an extended knapsack one.
Keywords: Iterative combination; iterative permutation; multi-set; parallel algorithm.

1. Introduction
The signi¯cant notions of combinatorics such as permutation, subset, arrangement,
combination and partition etc. are frequently used in many areas of computer science
such as scheduling problems, system controls, data mining and networking [1, 3, 4, 6,

10, 12, 13, 16, 21]. They become good tools to develop software and knowledge
engineering.
The notion of set was extended to multi-set. It is a set to which elements may
belong more than once. This extension sets up many new researches. Multi-sets were
employed in proving program correctness. It is often used in information processing
[7, 9, 15] and other problems. The well-known notions of combinatorics are being
transformed for multi-sets.
In this paper we ¯rst recall the notion of multi-set. Then we extend the notion
of permutation into iterative permutation, investigate its properties and construct an
e±cient algorithm to generate all iterative permutations of a multi-set. Applying the
parallelizing technique based on the output decomposition presented in [8, 19, 20] we
parallelize this algorithm. To do that, the sequence of desirable iterative permutations
of a multi-set is divided into sub-sequences with \nearly" equal lengths by appropriate choice of pivots. Using a common program (algorithm) with the corresponding
81


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

82

H. C. Thanh

input and termination condition, processors will execute in parallel to generate iterative permutations of these sub-sequences. Further, we apply the algorithm presented above for ¯nding the optimal solution of a work arrangement problem.
Analogously, we extend the notion of subset into multi-subset and combination
into iterative combination. Some new e±cient algorithms for generating all multisubsets, k-element multi-subsets and iterative k-combinations of a multi-set are
constructed and parallelized. Then we apply the algorithms to an extended knapsack
problem.
The newly constructed algorithms are based on our inheritance principle and
bounded sequences generation. These algorithms take part in developing combinatorial computing, software technology and especially in knowledge engineering

[5, 11]. They refer to the building and development of knowledge-based systems.
The rest of this paper is organized as follows. In Sec. 2 we recall the notion of
multi-set. Section 3 presents the notion of iterative permutation and constructs a
new e±cient algorithm based on lexicographical order for generating all iterative
permutations of a multi-set and then parallelizes the algorithm. Section 4 is devoted
to multi-subsets, an algorithm generating multi-subsets and its parallelism. Section 5
presents iterative combinations of a multi-set with generation algorithm. Using
algorithms presented above, two optimization problems are solved in Sec. 6. Finally,
some directions for future research are proposed in the conclusions.
2. Multi-sets
2.1. De¯nition of multi-set
Multi-set is an extended notion of standard set [9, 15] and is de¯ned as follows.
De¯nition 1. A multi-set is an unordered collection of elements in which elements
are allowed to be repeated.
For example, a set of values of all variables in a program, a marking of a net
system [14] etc., are typical examples of multi-sets.
A multi-set is written as follows:
X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ;

where ki ! 0 with i ¼ 1; 2; . . . ; n:

It means, there are k1 elements x1 , k2 elements x2 , … and kn elements xn in the multiset X.
Elements x1 ; x2 ; . . . ; xn are called basic elements of the multi-set X, and
k1 ; k2 ; . . . ; kn as the multiplicity of the corresponding element.
The cardinality of a multi-set is the sum of multiplicities of all its basic elemens,
n
X
jXj ¼
ki :
i¼1

A multi-set is called regularif all of its element' multiplicities are the same.


Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

83

2.2. Operations on multi-sets
Let A ¼ ðt1 à x1 ; t2 à x2 ; . . . ; tn à xn Þ and B ¼ ðq1 à x1 ; q2 à x2 ; . . . ; qn à xn Þ be two
multi-sets.
2.2.1. Multi-subsets
De¯nition 2. We say that A is a multi-subset of B i® the multiplicity of each
element in A is not greater than the multiplicity of the element in B, i.e.

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

A  B , 8 i ¼ 1; 2; . . . ; n : ti

qi :

Thus, each multi-subset A ¼ ðt1 à x1 ; t2 à x2 ; . . . ; tn à xn Þ of the multi-set X ¼
ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ must satisfy: 0 ti ki ; 8 i ¼ 1; 2; . . . ; n.
2.2.2. Power multi-set
De¯nition 3. The power multi-set of a given multi-set X, denoted by ðXÞ, is the
set of all multi-subsets of X.
2.2.3. Union
De¯nition 4. The union of two multi-sets A and B, denoted by A [ B, is the multiset C ¼ ðp1 à x1 ; p2 à x2 ; . . . ; pn à xn Þ, where pi ¼ maxðti ; qi Þ; 8 i ¼ 1; 2; . . . ; n.
2.2.4. Intersection

De¯nition 5. The intersection of two multi-sets A and B, denoted by A \ B, is the
multi-set D ¼ ðr1 à x1 ; r2 à x2 ; . . . ; rn à xn Þ, where ri ¼ minðti ; qi Þ; 8 i ¼ 1; 2; . . . ; n.
2.2.5. Di®erence
De¯nition 6. The di®erence of two multi-sets A and B, denoted by AnB, is
the multi-set H ¼ ðs1 à x1 ; s2 à x2 ; . . . ; sn à xn Þ, where si ¼ maxðti À qi ; 0Þ; 8 i ¼
1; 2; . . . ; n.
2.2.6. Lattice of multi-subsets
Including relation  on multi-subsets of a given multi-set X is a partial ordering.
For every two multi-subsets A and B in ðXÞ we de¯ne their supremum (sup) and
in¯mum (inf ) as follows.
De¯nition 7. For every two multi-subsets A and B in ðXÞ,
supðA; BÞ ¼ A [ B

and

infðA; BÞ ¼ A \ B:

The power multi-set ðXÞ equipped two operations sup and inf becomes a distributed lattice.


84

H. C. Thanh

3. Iterative Permutations
3.1. Notion of iterative permutation
We extend the notion of permutation of a multi-set as follows.
De¯nition 8. An iterative permutation of a multi-set is a checklist of all its elements.
Consider the following example.

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Example 1. Given a multi-set X ¼ ð2 Ã a; 1 Ã b; 1 Ã cÞ.
Iterative permutations of X are sorted by lexicographical order in Table 1.
It is easy to show that the number of all iterative permutations of a multi-set
X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ is cl ¼ k1 !k2l!!...kn ! , where l is the cardinality of X.
Table 1. Iterative permutations
of the multi-set X.
No

Iterative permutations

1

aabc

2

aacb

3

abac

4

abca

5

acab

6

acba

7

baac

8

baca

9

bcaa

10

caab

11

caba

12

cbaa

3.2. Iterative permutation generation algorithm
Iterative permutation problem
Given a multi-set X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ, with ki ! 1ði ¼ 1; 2; . . . ; nÞ.
Find all iterative permutations of X.
In other words, we have to construct an e±cient algorithm for generating all
iterative permutations of this multi-set.
It is easy to see that bijection is not suitable to represent iterative permutation.
So we have to ¯nd another representation.
Identify the elements x1  1; x2  2; . . . ; xn  n. Each iterative permutation of the
multi-set X is represented by an integer sequence of the length l, consisting of k1
integers 1, k2 integers 2, … and kn integers n. Now we ¯nd all these integer sequences.


Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

85

Each integer sequence may be considered as a word on the alphabet f1; 2; . . . ; ng.
So we sort these words increasingly by lexicographical order.
— The ¯rst word (the least) is 11
. . . 1ffl} 22
. . . 2ffl} . . . |fflfflfflffl{zfflfflfflffl}
nn . . . n . It is a non-decreasing
|fflfflfflffl{zfflfflffl
|fflfflfflffl{zfflfflffl
k1

k2

kn

sequence.
— The last word (the most) is |fflfflfflffl{zfflfflfflffl}
nn . . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . 11
. . . 1ffl}. It is a non|fflfflfflffl{zfflfflffl
kn

k1

knÀ1

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

increasing sequence and is indeed the reverse of the ¯rst one.
Starting with the ¯rst iterative permutation, our algorithm repeats a loop to ¯nd
the remaining iterative permutations.
Assume that t ¼ a1 a2 . . . alÀ1 al is an iterative permutation. We have to ¯nd the
iterative permutation t 0 ¼ a 01 a 02 . . . a 0lÀ1 a 0l next to t in the sorted sequence.
Because of lexicographic order, the iterative permutation t 0 is inherited the longest left part possible of the iterative permutation t from the ¯rst component to the
component indexed pÀ1; where the changing position p is the maximal index i, such
that ai < aiþ1 . Thus: p ¼ maxfij1 i l À 1 ^ ai < aiþ1 g.
(p)
t

a1

...

ap-1

ap

ap+1

...

t’ a1

...

ap-1

a’p

a’p+1

...

al
a’l

Fig. 1. Inheritance of iterative permutations.

It is easy to see that the sub-sequence hapþ1 ; apþ2 ; . . . ; al i is a non-increasing
sequence. The changing part in the iterative permutation t 0 from the position p to the
last is determined as follows:

1) a 0p is the least among all elements in the subsequence hapþ1 ; apþ2 ; . . . ; al i but
greater than ap . We swap ap for this element:
ap $ minfai jp þ 1

i

l ^ ai > ap g

Note that if there are several least elements we choose the element with the
greatest index. It ensures that after swapping the sub-sequence hapþ1 ; apþ2 ; . . . ; al i
still is a non-increasing sequence.
2) Reverse the sub-sequence hapþ1 ; apþ2 ; . . . ; al i. The result is a non-decreasing
sequence. It is indeed the sub-sequence ha 0pþ1 ; a 0pþ2 ; . . . ; a 0l i in the iterative permutation t 0 .
Our algorithm terminates when the last iterative permutation was generated. At
that time, the changing position p is equal to 0. We have Algorithm 1.


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

86

H. C. Thanh

Complexity of the algorithm:
— Instructions 2–3 input data with the complexity O(nÞ.
— Instructions 4–5 calculate the number of elements and assign the ¯rst iterative
permutation with the complexity O(lÞ.
— Loop 7–14 computes and prints an interative permutation, where: Instruction 7
prints an interative permutation with the complexity O(lÞ. Loop 9 locates the

changing position p with the complexity O(lÞ. Instructions 11–13 ¯nd the element a 0p and swap it for ap with the complexity O(lÞ. Instruction 14 reverses the
sub-sequence hapþ1 ; apþ2 ; . . . ; al i with the complexity O(lÞ.
So the complexity of generating an iterative permutation is O(lÞ. The total
complexity of Algorithm 1 is O(cl :lÞ.

3.3. Parallel algorithm generating iterative permutations
For the iterative permutation problem, we know the number of all desirable solutions
and their arrangement in the sequence sorted by lexicographic order. So we can
split the sequence into sub-sequences and use a common program (algorithm) in a
parallel computing environment to ¯nd concurrently the sub-sequences. Therefore,


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

87

Fig. 2. The model of the parallel computing organization to ¯nd the problem's solutions.

the amount of time required for ¯nding all desirable solutions will be drastically
decreased by the number of sub-sequences.
To perform the above parallel computing we split the sequence of all desirable
solutions into sub-sequences. The number of sub-sequences depends on the number of
calculating processors.
Let us split the sequence of all solutions of the problem into m sub-sequences
(m ! 2). The model of the parallel computing organization to ¯nd the problem's all
solutions is illustrated as in Fig. 2.
This model is an illustration of the following Condition/Event system [14, 17]:

Fig. 3. The Condition/Event system representing the above parallel computing model.

It is clear that events e1 ; e2 ; . . . ; em can be executed concurrently in this system.
In order to make the parallel computing organization realistic and optimal, the
sub-sequences of solutions should be satis¯ed for the following two criteria:
(1) It is easy to determine the input and the termination condition for the computing process of each sub-sequence.
(2) The di®erence of lengths of the sub-sequences is the less, the better.


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

88

H. C. Thanh

Of course, Input 1 is indeed the input of the problem and last termination condition is the termination condition of the algorithm. The ¯rst criterion ensures that
the splitting of solutions sequence is realistic. In many cases, we can use (a part of)
the input of the computing process for the next sub-sequence as the termination
condition of the computing process for the previous sub-sequence. The second
criterion implements the balance of computing to processors. Then the parallel
computing processes become optimal.
Applying the parallelizing technique presented above we parallelize Algorithm 1.
To do so, we split the sequence of all desirable iterative permutations of a multi-set
into m sub-sequences (m ! 2), such that lengths of the subsequences are \nearly"
equal. The number m depends on the number of processors the computing system
devotes to computing. Each sub-sequence will be generated by one computing
process executed on one processor. So input and termination condition for each
computing process must be already determined.

This parallelizing method is an illustration of the output decomposition technique
in parallel computing [8, 19, 20].
For simplicity of presentation we perform with m ¼ 3. With greater values of m
one can do analogously:
Assume that n ! 3. This assumption makes our splitting realistic. We split
the sequence of all sorted diserable iterative permutations into 3 sub-sequences
with \nearly" equal lengths by appropriate choice of the two following basic
elements:
( 
)!
q
X
l

r ¼ max 1; max q ki
and
 i¼1
3
( 
)!
q
X
l

s ¼ max r þ 1; max q
:
k
i¼rþ1 i 3
The ¯rst pivot chosen is the ¯rst iterative permutation of a block, whose ¯rst
element is r þ 1. It is:

r þ 1 |fflfflffl
11ffl{zfflfflffl
. . . 1ffl} 22
. . . 2ffl} . . . rr
. . . r r|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
þ 1r þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r þ 1ffl} r|fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}
þ 2r þ 2r þ 2 . . . nn
...n :
|fflfflfflffl{zfflfflffl
|fflfflffl{zfflfflffl}
|fflfflfflffl{zfflfflfflffl}
k1

k2

kr

krþ1À1

krþ2

kn

It becomes the input of the second computing process. The second pivot chosen is
the ¯rst iterative permutation of a block, whose ¯rst element is s þ 1. It is:
s þ 1 |fflfflffl
11ffl{zfflfflffl
. . . 1ffl} 22
. . . 2ffl} . . . ss

. . . s s|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
þ 1s þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s þ 1ffl} s|fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}
þ 2s þ 2s þ 2 . . . nn
...n :
|fflfflfflffl{zfflfflffl
|fflfflffl{zfflfflffl}
|fflfflfflffl{zfflfflfflffl}
k1

k2

ks

ksþ1À1

ksþ2

kn

and it becomes the input of the third computing process. The last iterative permutation of the ¯rst sub-sequence is indeed the last in a block, whose ¯rst element is r.


Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

89

Then it is the following iterative permutation:
r |fflfflfflffl{zfflfflfflffl}
nn . . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl

n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . r|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
þ 1r þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r þ 1ffl}
kn

knÀ1

krþ1

 rr
. . . r r|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
À 1r Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r À 1ffl} . . . 11
. . . 1ffl} :
|fflfflfflffl{zfflfflffl
|fflfflffl{zfflfflffl}

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

krÀ1

k1

krÀ1

So the termination for the ¯rst computing process is: A½1Š ¼ r ^ p ¼ 1.
The last iterative permutation of the ¯rst sub-sequence is indeed the last in a
block, whose ¯rst element is s. It is just the following iterative permutation:

s |fflfflfflffl{zfflfflfflffl}
nn . . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . s|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
þ 1s þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s þ 1ffl}
kn

knÀ1

ksþ1

 ss
. . . s s|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
À 1s Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s À 1ffl} . . . 11
. . . 1ffl} :
|fflfflfflffl{zfflfflffl
|fflfflffl{zfflfflffl}
ksÀ1

k1

ksÀ1

Thus, the termination for the second computing process is: A½1Š ¼ s ^ p ¼ 1.
The sequence of all iterative permutations of the multi-set X is divided into three
sub-sequences as in Table 2.
These three sub-sequences are generated concurrently by three parallel computing
processes, with the corresponding input and termination condition, executed

Table 2. Three sub-sequences of iterative permutation of the multi-set X.
Subsequence

Iterative permutations
. . . 2ffl} . . . ii
. .ffli} . . . nn
...n
11ffl{zfflfflffl
. . . 1ffl} 22
|fflfflfflffl{zfflfflffl
|fflfflffl
|fflfflffl.{zfflffl
|fflfflfflffl{zfflfflfflffl}

1

k1

k2

ki

kn

ÁÁÁ
r nn
. . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl

r þ 1r þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r þ 1ffl} |fflfflffl{zfflfflffl}
rr . . . r |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
r À 1r Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r À 1ffl} . . . |fflfflffl
11ffl{zfflfflffl
. . . 1ffl}
|fflfflfflffl{zfflfflfflffl}
kn

knÀ1

krÀ1

krþ1

k1

krÀ1

r þ 1 11
. . . 1ffl} |fflfflffl
22ffl{zfflfflffl
. . . 2ffl} . . . rr
. . . r |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
r þ 1r þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . r þ 1ffl} |fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}
r þ 2r þ 2r þ 2 . . . nn
...n
|fflfflfflffl{zfflfflffl

|fflfflffl{zfflfflffl}
|fflfflfflffl{zfflfflfflffl}

2

k1

k2

krþ1À1

kr

kn

krþ2

ÁÁÁ
s nn
. . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
s þ 1s þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s þ 1ffl} ss
. . . s |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
s À 1s Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s À 1ffl} . . . 11
. . . 1ffl}
|fflfflfflffl{zfflfflffl
|fflfflfflffl{zfflfflfflffl}

|fflfflffl{zfflfflffl}
kn

3

knÀ1

ksÀ1

ksþ1

k1

ksÀ1

s þ 1 |fflfflffl
11ffl{zfflfflffl
. . . 1ffl} |fflfflffl
22ffl{zfflfflffl
. . . 2ffl} . . . ss
. . . s |fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
s þ 1s þffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . s þ 1ffl} |fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}
s þ 2s þ 2s þ 2 . . . nn
...n
|fflfflffl{zfflfflffl}
|fflfflfflffl{zfflfflfflffl}
k1

k2

ksþ1À1

ks

ksþ2

ÁÁÁ
nn . . . n |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
n À 1n Àffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl
1 . . . n À 1ffl} . . . |fflffl
iiffl.{zfflffl
. .ffli} . . . |fflfflffl
11ffl{zfflfflffl
. . . 1ffl}
|fflfflfflffl{zfflfflfflffl}
kn

knÀ1

ki

k1

kn


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

90
H. C. Thanh

concurrently on three processors P1 , P2 and P3 as follows:


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

91

So the time for generating all iterative permutations of a multi-set reduces to onethird.
4. Multi-Subset Problem
As mentioned above, each multi-subset A ¼ ðt1 à x1 ; t2 à x2 ; . . . ; tn à xn Þ of the multiset X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ must satisfy: 0 ti ki ; 8 i ¼ 1; 2; . . . ; n. So
we have the following problem.
Problem: Given a multi-set X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ. Find all multi-subsets
of X.
Q
The number of all multi-subsets of the multi-set X is ni¼1 ðki þ 1Þ. A multi-subset
A ¼ ðt1 à x1 ; t2 à x2 ; . . . ; tn à xn Þ may be represented by the integer sequence
ht1 t2 . . . tn i.
4.1. Bounded sequence algorithm generating multi-subsets
In order to solve the multi-subset problem we recall the bounded sequence problem
presented in [18] as follows:
Given two integer sequences of the length n; s ¼ hs1 s2 . . . sn i and g ¼ hg1 g2 . . . gn i,
such that s g. Find all integer sequences t ¼ ht1 t2 . . . tn i with the length of n,
satisfying s t g.

92

H. C. Thanh

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

The multi-subset problem may be derived to the bounded sequence problem with
two boundary sequences s ¼ h00 . . . 0i and g ¼ hk1 k2 . . . kn i.
Applying the algorithm generating bounded sequences presented in [18] we have
the following short algorithm for the multi-subset problem.

Complexity of the algorithm:
Loop 7 ¯nds the changing position i with the complexity OðnÞ. So the complexity
Q
of Algorithm 2. is Oðn: ni¼1 ðki þ 1ÞÞ.
4.2. Parallel algorithm generating multi-subsets
Applying the parallelizing technique presented above, we split the sequence of all
multi-subsets into m sub-sequences.
Assume that m k1 þ 1. We divide the integer segment [0, k1 ] into m \nearly
equal" sub-segments.
Let denote p ¼ ðk1 þ 1Þ div m and q ¼ ðk1 þ 1Þ mod m.
The processor jðj ¼ 1; 2; . . . ; mÞ ¯nds all multi-subsets, where the multiplicity t1
belongs to the sub-segment j. Hence, we assign new values for S[1] and G[1] as follows:
&
ðj À 1Þ Á ðp þ 1Þ; if j q
S½1Š ¼
q þ ðj À 1Þ Á p;
otherwise

and

&
G½1Š ¼

S½1Š þ p;
if j q
:
S½1Š þ p À 1; otherwise

The algorithm for the processor jðj ¼ 1; 2; . . . ; mÞ is the following.


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

93

The complexity of Algorithm 3
The length of a sub-sequence of multi-subsets computedÀ by the processor
Á j is near
Q
Qn
n
to m1 ni¼1 ðki þ 1Þ. So the complexity of Algorithm 3 is O m
ðk
þ
1Þ

.
i¼1 i
5. Iterative Combinations
Let X be a multi-set and k be a non-negative integer. We extend the notion of
combination on multi-sets as follows.
De¯nition 9. An iterative k-combination of the multi-set X is a selection of k
elements from the multi-set X.
Thus, each iterative k-combination of the multi-set X is indeed a k-element multisubset of X.
Iterative combination problem. Given a multi-set X. Find all iterative k-combinations of the multi-set X.
The problem may be derived to ¯nding all k-element multi-subsets of X.
Assume that X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ, where ki ! 0 with i ¼ 1; 2; . . . ; n.
An k-element multi-subset of X; A ¼ ðt1 à x1 ; t2 à x2 ; . . . ; tn à xn Þ must satisfy the
following system of linear in-equations:
&
0 ti ki ; i ¼ 1; 2; . . . ; n
:
ð1Þ
t1 þ t2 þ Á Á Á þ tn ¼ k
Denote ck as the number of all k-element multi-subsets
 of X.

nþkÀ1
As shown in [12], if k1 ¼ k2 ¼ Á Á Á ¼ kn ¼ k then ck ¼
.
k


94

H. C. Thanh

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

In the general case, for any multiplicity ki > k we may replace ki by k but ck will not
be changed. So without loss of generality, we can assume that: ki k with i ¼
1; 2; . . . ; n. In this case we solve the system of in-equations (1) to ¯nd all iterative
k-combinations of X. But solving (1) is not easy. Thus, we have to detect other methods.
Theorem 1. The number of all iterative k-combinations of the multi-set X ¼
ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ is:

 X

n 
nþkÀ1
n þ k À ki À 2
ck ¼
À
k
k À ki À 1
i¼1


X
n þ k À ki À kj À 3
À Á Á Á þ ðÀ1Þ n
þ
k
À
k

À
k
À
2
i
j
1 i

n þ k À k1 À k2 À Á Á Á À kn À n À 1
Â
:
k À k1 À k2 À Á Á Á À kn À n
Proof. Denote Z as the set of all k-element multi-subsets of the regular multi-set


nþkÀ1
X ¼ ðk à x1 ; k à x2 ; . . . ; k à xn Þ. The set Z consists of
multi-subsets. All
k
k-element multi-subsets of X belong to Z.
To calculate the number of all k-element multi-subsets of X we remove all multisubsets in Z, which are not k-element multi-subset of X. Each k-element multi-subset
of X must satisfy all n in-equations in (1). If an k-element multi-subset does not
satisfy only one in-equation, the multi-subset is not k-element multi-subset of X.
Let Zi denote the set of all multi-subsets in Z, where the multiplicity
ti > ki ; ði ¼ 1; 2; 3; . . . ; nÞ. The set of all multi-subsets in Z, which are not k-element
S
multi-subset of X, is ni¼1 Zi .
Applying the inclusion-exclusion principle for sets [12], we have:








[
n
n
[




ck ¼ Z À Zi  ¼ jZj À  Zi 




i¼1
i¼1
!
n
X
X
nÀ1
¼ jZj À
jZi j À
jZi \ Zj j þ Á Á Á þ ðÀ1Þ jZ1 \ Z2 \ Á Á Á \ Zn j :
i¼1

1 i
Calculating cardinalities of Zi and their intersections we ¯nd out the number of
all k-element multi-subsets of X to be:


 X
n 
n þ k À ki À 2
nþkÀ1
ck ¼
À
k À ki À 1
k
i¼1
X  n þ k À ki À kj À 3 
þ
À Á Á Á þ ðÀ1Þ n
k À ki À kj À 2
1 i

n þ k À k1 À k2 À Á Á Á À kn À n À 1
Â
:
k À k1 À k2 À Á Á Á À kn À n
This proves the theorem.

Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

95

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Note that if there is a negative parameter in some combination, we ignore the
combination.
Now we construct an algorithm to generate all iterative k-combinations of
a multi-set. As mentioned above, each iterative k-combination of the multi-set X
is indeed a k-element multi-subset of X. Using Algorithm 2. for generating all
multi-subsets of X and printing only k-element multi-subsets, we have the following
algorithm generating all iterative k-combinations of the multi-set X.

Complexity of the algorithm
Instructions 6À7 ¯nd a new sequence ht1 t2 . . . tn i with the complexity of
O(nÞ. Instructions 8À9 check whether the sequence ht1 t2 . . . tn i is an iterative
k-combination of the multi-set X with the complexity of O(n). So the complexity
Q
of Algorithm 4. is Oðn: ni¼1 ðki þ 1ÞÞ.

6. Some Applications
6.1. Optimal work arrangement
Assume that we have k1 works e1 ; k2 works e2 ; . . . and kn works en to be performed.
These works require the same amount of time to perform, e.g. one day. The works
are performed uninterruptedly. Only one work is being performed at any moment.
For the work ei ði ¼ 1; 2; . . . ; nÞ, di is the deadline, xi is the reward money per day
for early ¯nish and yi is the penalty money per day for late ¯nish. Arrange these
works, such that the money obtained is the most possible.

This problem is an extensions of the work arrangement problem presented in
[12, 21], which was solved by matroid. For our problem, we cannot create any matroid.


96

H. C. Thanh

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

P
Denote l ¼ m
i¼1 ki . We have l works. The set of works may be represented by the
following multi-set X ¼ ðk1 à e1 ; k2 à e2 ; . . . ; kn à en Þ.
Identify work e1  1; e2  2; . . . ; en  n. Each arrangement of the works is
indeed an iterative permutation of X. It is represented by an integer sequence
ha1 a2 . . . alÀ1 al i of the length l, with k1 integers 1, k2 integers 2, …and kn integers n.
Construct a reward/penalty function for the works as follows:
&
ðdi À jÞxi ; if aj and j < dt
tpðaj Þ ¼
ðdi À jÞyi ; if aj and j > dt
With an arrangement as the iterative permutation ha1 a2 . . . alÀ1 al i the total
money obtained is:
Tg ¼

l
X

tpðaj Þ:

j¼1

The optimal problem of work arrangement is derived to the problem of ¯nding
an iterative permutation with the most Tg.
Combining Algorithm 1 and the algorithm ¯nding the largest number in a
sequence of numbers we have an algorithm for optimal arrangement of works.
We may use the parallel algorithm presented in Sec. 3.3 to get the result more
quickly if the computing system allows.
Example 2. Given 3 works as in Table 3.
Table 3.

Three works and their data.

Work ei

Quantity ki

Deadline di

Reward xi

Penalty yi

1
2
3
4

2
1
2
3

6
3
5
2

7
4
6
9

3
2
4
5

We have a multi-set X ¼ ð2 Ã e1 ; 1 Ã e2 ; 2 Ã e3 ; 3 Ã e4 Þ. This multi-set has 1680
iterative permutations. They are all possible variants of the work arrangements.
Computing by the above algorithm we get the following optimal variant: 1 1 3 4 4
4 3 2, in which the largest possible money obtained is 12.
6.2. Extended knapsack problem
The extended knapsack problem is a natural generalization of the 0/1 knapsack
problem [2, 5] and is de¯ned as follows:
Given n precious items and a knapsack with the capacity c. Each item i has a
quantity ki , a pro¯t pi and a capacity si ði ¼ 1; 2; . . . ; nÞ. The goal is to ¯nd a subset
of items of maximum pro¯t such that they have a feasible packing in the knapsack.

The extended knapsack problem is probably one of the most interesting and
popular in computer science. When ki ¼ 1ði ¼ 1; 2; . . . ; nÞ the extended knapsack


Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

97

problem becomes the 0/1 knapsack problem. The 0/1 knapsack problem was
approximately solved by dynamic programming, backtracking and greedy method
etc. Using multi-subsets generation we can construct an exact algorithm to solve the
extended knapsack problem.
Denote the ¯rst item as x1 , the second one x2 , …and nth- item xn . The set of
items is represented by a multi-set X ¼ ðk1 à x1 ; k2 à x2 ; . . . ; kn à xn Þ. Each variant of
choosing items to pack in the knapsack is a multi-subset A ¼ ðt1 Ã x1 ; t2 Ã x2 ; . . . ; tn Ã
xn Þ of X, such that:
n
X
ti Á si c:
i¼1

P
An optimal variant is the variant, whose pro¯t ni¼1 ti Á pi becomes maximum.
Using Algorithm 2 to generate multi-subsets we get quickly an optimal solution of
the extended knapsack problem.
Example 3. Consider the extended knapsack problem with data as in Table 4
Table 4. Five items and their data.

Item

Quantity

Pro¯t

Capacity

a
b
c
d
e

2
4
6
3
5

12
8
5
7
9

6
3
7
4

5

and the capacity of the knapsack is 60.
Calculating using Algorithm 2 we get an optimal packing: A ¼ ð2 Ã a; 4 Ã b; 0 Ã c;
2 Ã d; 5 Ã eÞ where the sum of the capacities of the chosen items is 57 and the maximum pro¯t is 115.
The extended knapsack problem is often used in communication systems and
computer networks.
7. Conclusions
In this paper we extended some notions of combinatorics over multi-sets such as
iterative permutation, multi-subset and iterative combination. These notions have
been used in many areas of computer science, data mining, system controls and
knowledge engineering.
We constructed some new e±cient algorithms for generating iterative permutations, multi-subsets and iterative combinations of a multi-set. Applying the
parallelizing method based on output decomposition, we parallelized these algorithms
and applied them to solving two optimal combinatorial problems: the optimal work
arrangement and the extended knapsack problem. These parallel algorithms may be
used in communication, class-association rule mining and knowledge engineering etc.


98

H. C. Thanh

We keep investigating combinatorics over multi-sets and apply them to huge
computing problems in time-series data matching as well as in data mining.

Acknowledgments

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

The author would like to acknowledge Vietnam's National Foundation for Science
and Technology Development (NAFOSTED) as the sponsor for the research project
102.01-2012.17.

References
1. M. Barnabei, F. Bonetti and M. Silimbani, Two permutation classes related to the bubble
sort operator, Electronic Journal of Combinatorics, 19 (2012) 25.
2. D. Bertsimas and R. Demir, An approximate dynamic programming approach to multidimentional knapsack problem, Management Science 48 (2002) 550À565.
3. F. Chung, A. Claesson, M. Dukes and R. Graham, Descent polynomials for permutations
with bounded drop size, European Journal of Combinatorics 31 (2010) 1853À1867.
4. R. Ehrenborg, S. Kitaev and P. Perry, A spectral approach to consecutive patternavoiding permutations, Journal of Combinatorics 2 (2011) 305À354.
5. A. E. Eiben and J. E. Smith, Introduction to Evolutionary Computing (Springer-Verlag,
2010).
6. J. Ginsburg, Determining a permutation from its set of reductions, Ars Combinatoria 82
(2007) 55À57.
7. K. P. Girish and J. J. Sunil, General relations between partially ordered multi-sets and
their chains and antichains, Math. Commun. 14 (2009) 193À205.
8. A. Grama, A. Gupta, G. Karypis and V. Kumar, Introduction to Parallel Computing
(Addison-Wesley, 2003).
9. S. L. Hu and C. Y. Shi, A multi-subset possible world semantics for intention operator of
agent, Lecture Notes in Computer Science Vol. 4088, 2006, pp. 484À489.
10. T. Kuo, A new method for generating permutations in lexicographic order, Journal of
Science and Engineering Technology 5 (2009) 21À20.
11. S. L. Kendal and M. Creen, An Introduction to Knowledge Engineering (Springer, 2007).
12. W. Lipski, Combinatorics for Programmers (WNT, Warsaw, 1982) (in Polish).
13. T. Mansour and M. Shattuck, Set partitions as geometric words, The Australasian
Journal of Combinatorics 53 (2012) 31À40.
14. W. Reisig, Petri Nets: An Introduction (Springer-Verlag, 1985).
15. D. Singh, A. M. Ibrahim, T. Yohanna and J. N. Singh, An overview of the applications of

multi-sets, Novi Sad Journal of Mathematics 37 (2007) 73À92.
16. J. Steinhardt, Permutations with ascending and descending blocks, Electronic Journal of
Combinatorics 17 (2010) 14.
17. H. C. Thanh, Transforming sequential processes of a net system into concurrent ones,
International Journal of Knowledge-based and Intelligent Engineering Systems 11 (2007)
391À397.
18. H. C. Thanh, Bounded sequence problem and some of its applications, Proc. of
JapanÀVietnam Workshop on Software Engineering, Hanoi, 2010, pp. 74À83.
19. H. C. Thanh and N. Q. Thanh, An e±cient parallel algorithm for the set partition
problem, Studies in Computational Intelligence Vol. 351, 2011, pp. 25À32.


Parallel Combinatorial Algorithms for Multi-Sets and Their Applications

99

Int. J. Soft. Eng. Knowl. Eng. 2013.23:81-99. Downloaded from www.worldscientific.com
by QUEEN'S UNIVERSITY on 10/08/14. For personal use only.

20. H. C. Thanh, Parallel generation of permutations by inversion vectors, Proc. of IEEERIVF International Conference on Computing and Communication Technologies, IEEE,
2012, pp. 129À132.
21. Y. L. Wang, Y. L. Ueng, C. L. Peng and C. J. Yang, Processing-task arrangement for a
low-complexity full-mode WiMAX LDPC codec, IEEE Transactions on Circuits and
Systems 58 (2011) 415À428.