Tài liệu Module 5: Clustering pptx

Contents
Resource Dependencies 1
Cluster Service Account Permissions 5
MsExchange_NodeState 9
DNS registration/Kerberos 12
AntiAffinityClassNames 16
Mount Point Drives 22
Creating an Exchange Virtual Server 33
Upgrading an Exchange Virtual Server to
Exchange 2003 56
Removing an Exchange Virtual Server 64
Lab 5.1 : Clustering 88

Module 5: Clustering


Information in this document, including URL and other Internet Web site references, is subject to
change without notice. Unless otherwise noted, the example companies, organizations, products,
domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious,
and no association with any real company, organization, product, domain name, e-mail address,
logo, person, place or event is intended or should be inferred. Complying with all applicable
copyright laws is the responsibility of the user. Without limiting the rights under copyright, no
part of this document may be reproduced, stored in or introduced into a retrieval system, or
transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or

otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual
property rights covering subject matter in this document. Except as expressly provided in any
written license agreement from Microsoft, the furnishing of this document does not give you any
license to these patents, trademarks, copyrights, or other intellectual property.

 2003 Microsoft Corporation. All rights reserved.

Microsoft, MS-DOS, Windows, Windows NT, Active Directory, ActiveX, Excel, Exchange Server
5.5, Exchange 2000 Server, Exchange Server 2003, Internet Explorer, Internet Information Server,
Word are either registered trademarks or trademarks of Microsoft Corporation in the United States
and/or other countries.

The names of actual companies and products mentioned herein (Groupwise, Lotus cc:Mail, Lotus
Notes) may be the trademarks of their respective owners.


Module 5: Clustering 1


Resource Dependencies


In an Exchange 2000 cluster, we need to create a new Cluster Group to house
the Exchange Virtual Server. In order to successfully create a System Attendant
Resource, we must first have a physical disk resource, an IP address, and a
Network Name in that group.
When we create the System Attendant resource, the other Exchange resources
will be automatically created. During the creation process, a dependency tree

will be created. The dependency tree is shown below.



2 Module 5: Clustering


The Information Store resource has five dependencies: SMTP, HTTP, POP,
IMAP and Microsoft Search service. The message transfer agent (MTA) and
Routing Engine resources are directly dependant on the System Attendant. In
the event of a failover, all resources that have a dependency must go offline
before the resource that it is dependant on them can attempt to go offline.
In the scenario above the SMTP, HTTP, IMAP4, POP3 and Microsoft Search
service must successfully go offline (or fail) before the Information Store
resource can attempt to go offline. The MTA and Routing Engine resources can
attempt to go offline immediately, as they do not have any resources that are
dependant on them.
Traditionally in Exchange 2000 clusters, the SMTP and the Information Store
resources took the longest amount of time to go offline/come online. This could
be attributed to large SMTP queues or mounting/dismounting large databases.
This obviously will lead to longer failover times as the Information Store
resource has to wait for the SMTP resource to go offline before it can attempt to
go offline/come online.
Exchange 2000
Resource Dependency
Tree
Module 5: Clustering 3


In Exchange Server 2003, the resource-dependant tree has been altered so that

all Exchange 2003 cluster resources are now directly dependant on the System
Attendant resource.


Here we see that all the Exchange related resources are now directly dependant
on the System Attendant. This effectively means that the SMTP (and other
protocol resources) can now be brought online/go offline in parallel with the
store. This makes for faster failovers of the Exchange Virtual Server.
During the creation of the Exchange Virtual Server process, the correct
dependencies will be set.

The POP3 and IMAP4 resources are not created by default. If they are
created manually, then you will need to set a dependency on the System
Attendant (this is mandatory).

During an upgrade of an Exchange 2000 Exchange Virtual Server, the resource
dependencies will be changed to the new Exchange 2003 resource dependency
tree. From the “Exchange Server Setup Progress.log” file we can see these
changes being made. Open the log file and search for
ScUpgradeResourceDependencies. Here we will see each resource being
changed.
An SMTP resource being changed from the progress log:
Resource Dependency
Tree in Exchan
g
e 2003
Note
4 Module 5: Clustering

[08:36:54] Entering ScUpgradeResourceDependencies
[08:36:54] Checking dependencies of resource 'SMTP Virtual
Server Instance - (EVS-01)'
[08:36:54] Entering ScChangeResourceDependency
[08:36:54] About to change resource dependency for resource
'SMTP Virtual Server Instance - (EVS-01)'
[08:36:54] Leaving ScChangeResourceDependency

You will see the above entries for all Exchange resources that are upgraded to
Exchange 2003.
Module 5: Clustering 5


Cluster Service Account Permissions


Related articles/bugs:
 329702.KB.EN-US
In order to successfully create, delete or modify an Exchange 2000 Exchange
Virtual Server, the Windows 2000 cluster service account required “Exchange
Full Administrator” permissions at the organization level if it was the first
Exchange Virtual Server in the org. If it was not the first Exchange Virtual
Server in the org then it required Exchange Full Administrator on the Admin
Group that it was being installed into.

6 Module 5: Clustering

The Exchange Virtual Server creation process (shown above) can be broken
down as follows:

1. User DOMAIN\Administrator logs in to one of the Nodes and starts Cluster
Administrator (cluadmin.exe). The process cluadmin.exe runs as the
currently logged in user (DOMAIN\Administrator). The Administrator then
attempts to create a new Exchange System Attendant. Excluadmin.dll will
gather information from Active Directory in order to create the System
Attendant (e.g. Org name and Administrative Group name etc). The user
DOMAIN\Administrator needs permissions to read from the configuration
partition of the Active Directory.
2. When excluadmin.dll has collected the necessary information, it will then
pass the information to exres.dll. Exres.dll is the Exchange resource dll.
Exres.dll runs in the Resource Monitor process, which runs in the context of
the Cluster Service Account.
3. Exres.dll will then load exsetdata.dll in order to create the objects in Active
Directory. Exsetdata.dll also runs in the Resource Monitor process.
4. Exsetdata.dll will then create the necessary objects in the Active Directory.
As Exsetdata.dll runs in the context of the Cluster Service Account, this
account will require Full Exchange Administrator permissions in order to
create the objects successfully.
Permission
requirements in
Exchange 2000
Module 5: Clustering 7


In Exchange 2003 the permissions have changed in order to remove this
requirement. Any person or application that runs as the Windows 2000 cluster
service account essentially has the ability to destroy an Exchange 2000

organization.

The Exchange 2003 permissions requirements are as follows:


In the Exchange 2003 the Exchange Virtual Server creation process can be
broken down as follows:

1. The user DOMAIN\Administrator logs in to a Node in the cluster and starts
Cluster Administrator (cluadmin.exe). This process runs in the context of
DOMAIN\Administrator. The Administrator then attempts to create a new
Exchange System Attendant resource. Excluadmin.dll will gather
information from Active Directory in order to create the System Attendant
(e.g. Org name and Administrative Group name etc). The user
DOMAIN\Administrator will need to permissions to read from Active
Directory for this operation to be successful.
2. When excluadmin.dll has collected the necessary information, it will then
load Exsetdata.dll directly. Exsetdata.dll runs in the same process as
Excluadmin.dll (DOMAIN\Administrator).
3. Exsetdata.dll will then create the objects in Active Directory. As
exsetdata.dll runs in the context of DOMAIN\Administrator, it is this
account that requires the Exchange Full Administrator permissions to the
configuration partition of Active Directory.
Permissions
requirements in
Exchan
g
e 2003
8 Module 5: Clustering

After an Exchange 2000 Exchange Virtual Server has been successfully
upgraded to Exchange 2003 the cluster service account for that cluster can
be removed from the organization and/or Administrative Group objects’
permissions using the delegate control wizard. Remember that if that
account is used by other Exchange 2000 clusters, then you will have to
leave the permissions in place until they have been upgraded to Exchange
2003
Windows 2000 Cluster Service Account:
 Local Administrator on each Node in the cluster
 Exchange Full Administrator on org object if other Exchange 2000 clusters
remain in org

Windows 2003 Cluster Service Account
 Local Administrator on each Node
 No permissions required on org

Permissions required
quick check:
Module 5: Clustering 9


MsExchange_NodeState

MsExchange_NodeState is a registry value that is set during the installation of
the Exchange 2003 binary files on a cluster Node. It can be seen in the
following screen shot:

In this screen shot we can see that Nodes 1 and 2 have the value set and Nodes
3 and 4 do not.

In Exchange 2000 we did not check if a node in the cluster had the Exchange
binary files installed. This would cause problems when calculating if
Active/Active or Active/Passive was allowed in the cluster.
For example, If we have a three-Node cluster but we have only installed
Exchange 2000 on two of the Nodes, we should be allowed to have an
10 Module 5: Clustering


Active/Active or Active/Passive cluster, as only two of the Nodes can actually
host an Exchange Virtual Server. Unfortunately exres.dll (The Exchange cluster
resource .dll) does not see that one of the Nodes does not have the Exchange
2000 binaries and therefore should not be counted when calculating if
Active/Active or Active/Passive is allowed. In this scenario if we created two
Exchange Virtual Servers and then tried to bring both of them online on one
Node, one of the Exchange Virtual Servers would fail to come online.

In Exchange 2003 we now write this value to the registry on each Node in order
to tell us that the Node is Exchange enabled (i.e. it can be added as a possible
owner of an Exchange 2003 resource).
It is also used to determine the maximum number of Exchange Virtual Servers
that can be created in the cluster.

For example if we have six Nodes in the cluster but only four of them have the
MSExchange_NodeState set to 1, then we can create a maximum of three
Exchange Virtual Servers.
The maximum is three in this example, as we always enforce at least one
passive node in cluster with greater than two Nodes.

The registry value is written to the following location:
HKLM\Cluster\Nodes\<Node_ID>\Parameters\MSExchange_NodeState
REG_DWORD=1


The Node_ID value is set when the node is added as a member of the
cluster. The first node in a cluster will always have a Node ID of 1.

When the value is set to 1, then the node is Exchange 2003-enabled and can be
set as a possible owner of Exchange 2003 resources in the cluster.
If this value is set to 0 or does not exist, then the Node will not be allowed as a
possible owner for Exchange 2003 resources.
If you attempt to add a cluster node as a possible owner of an Exchange 2003
resource you will receive the following error:

Another registry entry MSExchange_CurrentBuild is used to track the version
of Exchange 2003 that is installed on the Node. For example, if we were to
upgrade the binary files on Node 1 to a higher value than Node 2, then the
MSExchange_CurrentBuild versions would be different on the two nodes.
Note
Module 5: Clustering 11


From the Exchange Server Setup Progress log we can see Setup writing these
attributes:
[02:25:13] Entering
CAtomClusterServer::ScSetExchangeStateOnCluster
[02:25:13] Entering CAtomClusterServer::ScSetNodeProperty
[02:25:13] Setting DWORD MSExchange_NodeState=1 on node
'NODE1'

[02:25:13] Setting DWORD MSExchange_CurrentBuild=452526080 on
node 'NODE1'
[02:25:13] Leaving CAtomClusterServer::ScSetNodeProperty
[02:25:13] Leaving
CAtomClusterServer::ScSetExchangeStateOnCluster
[02:25:13] Entering
CAtomClusterServer::ScEnableNodeAsPossibleOwer
[02:25:13] Leaving
CAtomClusterServer::ScEnableNodeAsPossibleOwer

This can also be seen from the cluster log:
00000550.00000308::2003/04/20-00:25:13.497 INFO [DM] Setting
value of MSExchange_NodeState for key Nodes\1\Parameters to
0x00000001

00000550.00000308::2003/04/20-00:25:13.497 INFO [DM] Setting
value of MSExchange_CurrentBuild for key Nodes\1\Parameters to
0x1af90000

12 Module 5: Clustering


DNS registration/Kerberos


Related articles:
 Article 235529
Windows 2000 SP3 added support for Kerberos authentication against clustered
virtual servers. Prior to Windows 2000 SP3, all authentication against clustered
virtual servers was NTLM or NTLM version 2 (NTLMv2). Prior to Windows

2000 SP3, a clustered virtual server did not have a corresponding Active
Directory computer object. When the RequireKerberos property was set to 1
and the Network Name resource was restarted an Active Directory computer
object would be created.
Support for another property RequireDNS was also added in SP3. This would
mean that the clustered virtual server would have to successfully register its
own host record (A-record) in DNS in order to come online.
Both of these properties are private properties of a Network Name resource and
can be set to either 0 or 1. RequireKerberos and RequireDNS have defaults of 1
and 0, respectively. Setting either of these values to 1 is not supported for an
Exchange 2000 Virtual Server. An Exchange 2000 Network Name resource
required that these properties be set to 0.
Therefore all authentication against Exchange 2000 Virtual Servers is NTLM or
NTLM version 2.
In Exchange 2003 we now enforce the use of Kerberos authentication. This is
done automatically by the setup process for non-clustered servers. For clusters,
these private properties are set during the creation of the virtual server. To view
the properties in Windows 2000 we need to use the command line cluster.exe
tool as follows:
Cluster res “my EVS Network Name” /priv

Windows 2000 SP3
Module 5: Clustering 13


In Windows 2000 this can only be set by using the command line tool
cluster.exe. In the screenshot above, the cluster.exe command has already been
used to change the RequireDNS property to a value to “1”.
14 Module 5: Clustering

In Windows 2003 Server these properties are changeable from the GUI of
cluster administrator.

As mentioned earlier, there is no need to set these manually as the setup process
will automatically set requireKerberos to 1 and requireDNS to 0. If, after
starting these resources with their defaults, you change requireKerberos to 0
(i.e. uncheck it from the UI,) then the Network Name resource will fail to come
online. Changing the DNS requirement does not affect how resources come
online unless you are using static DNS, where all records need to be entered by-
hand.
From the Exchange Server Setup Progress.log we can see Exchange Virtual
Server-creation setting these values:

Module 5: Clustering 15


[07:23:58] Entering ScEnableKerberosAndDNSOnEVS
[07:23:58] Entering ScBindToEVS
[07:23:58] Binding to cluster'node02.exchange.org'
[07:23:58] Binding to EVS 'EVS01'
[07:23:58] Leaving ScBindToEVS
[07:23:58] Searching for network name resource 'EVS01'
[07:23:58] Resource 'EVS01 Network Name' owns network name
'EVS01'
[07:23:58] Entering ScEnableKerberosAndDNSOnResource
[07:23:58] Entering ScGetKerberosAndDNSFromNetworkNameResource
[07:23:58] RequireKerberos=0
[07:23:58] RequireDNS=0
[07:23:58] Leaving ScGetKerberosAndDNSFromNetworkNameResource

[07:23:59] Setting properties to enable Kerberos on resource
'EVS01 Network Name'
[07:23:59] Entering ScSetKerberosAndDNSPropertiesOnResource
[07:23:59] Setting property 'RequireKerberos'='1'
[07:23:59] Leaving ScSetKerberosAndDNSPropertiesOnResource
[07:23:59] Bringing resource 'EVS01 Network Name' online
[07:24:39] Network name resource 'EVS01 Network Name'
successfully enabled for Kerberos
[07:24:39] Entering BringDependentResourcesOnline
[07:24:39] Bringing online resources dependent on network
name resource
[07:24:39] Leaving BringDependentResourcesOnline
[07:24:39] Leaving ScEnableKerberosAndDNSOnResource
[07:24:39] Leaving ScEnableKerberosAndDNSOnEVS

The requirement for DNS registration to succeed was removed from Exchange
2003 in build 6917. This was due to the fact that the Network Name resource
would fail to come online if the DNS service in use did not support dynamic
updates.
If a dynamic update DNS service is in use, then the RequireDNS can be set to
1 for an Exchange 2003 Network Name resource.
If a non dynamic update DNS service is in place then it is essential to make sure
that the correct A-records exist for the Exchange Virtual Server Network
Names resource. If this is missing you may run into transport/routing issues.
The requireKerberos setting will now create an Active Directory object for the
Exchange Virtual Server. In order for this to succeed, the Cluster Service
account will need to have the “Add workstations to the Domain” right. It should
be a domain account and will therefore have this right by default.
A detailed description of the Network Name resources in Windows 2003 can be
obtained in article 302389.

16 Module 5: Clustering


AntiAffinityClassNames


AntiAffinityClassNames is a new feature of Windows 2003 clustering. It gives
us the ability to assign a node as a hot spare for a particular cluster group in a
cluster of three or more Nodes.
AntiAffinityClassNames is a public property of cluster groups in Windows
2003 that contains an arbitrary string of characters.

AntiAffinityClassNames is not available in Windows 2000 clusters.

If there is a failure of a resource in a Windows 2003 cluster group (and the
resource is configured to affect the group), the failover manager will check the
value of the AntiAffinityClassNames attribute. It will then look for a Node in
the cluster that does not host a cluster group with that AntiAffinityClassNames
value. If there is such a node in the cluster, then it is considered to be the best
possible destination for the cluster group.

Note
Module 5: Clustering 17


In the image above we can see a four-Node cluster. There are three Exchange
Virtual Servers and the AntiAffinityClassNames property is set to “Microsoft
Exchange Virtual Server”.
If EVS01 was to fail, then failover manager would try and find a node in the
cluster that did not own a cluster group with this property set to “Microsoft

Exchange Virtual Server”.
In this case EVS01 would fail over to node CLU-NODE04.

This, of course, assumes that node CLU-NODE04 is set as a possible
owner of the resources that make up EVS01 (which it should be by default).

Note
18 Module 5: Clustering


To check the value of AntiAffinityClassNames use the cluster.exe from the
command prompt.
Cluster group “my Group” /prop

The value is blank by default.
To set the property on a cluster group use the following syntax:
Cluster group “my Group” /prop
AntiAffinityClassNames=”Some_Text_String”

The value of AntiAffinityClassNames is not visible from the GUI. It can only
be seen using the cluster.exe command line tool.
Module 5: Clustering 19


When one creates an Exchange 2003 virtual server on a Windows 2003 cluster
this attribute will be automatically set to “Microsoft Exchange Virtual
Server”. If you are seeing it set to some other string then it has probably been
changed manually and should be changed back to the default setting.

20 Module 5: Clustering

Using Cluster Administrator, we can manually move a group to another Node
in the cluster. When we have a cluster with more than two Nodes, we will have
a new choice called Best Possible.

In the above image we can see that there are three Nodes in the cluster. By
selecting Best Possible, then the cluster group containing the Exchange Virtual
Server will be moved to a Node that does not currently host a group with the
AntiAffinityClassNames set to “Microsoft Exchange Virtual Server”.
Module 5: Clustering 21


It is important to note that the AntiAffinityClassNames property is set on the
cluster group. If we were to create a new cluster group and then move all of the
Exchange 2003 cluster resources into this group then the attributes that were set
on the original cluster group do not follow the resources.
The attributes that are affected in this scenario are AntiAffinityClassNames and
MSExchange_VirtualServerNames.
It is unsupported to move the Exchange 2003 cluster resources between cluster
groups. If you are not seeing these properties set on the Exchange 2003 cluster
group then the resources may have been moved manually.
In this scenario it is necessary to recreate the attributes using the command line
cluster.exe utility
Related articles:
Windows 2000 DataCenter Server:
 279802 – The default behaviour when you do a move group operation on a
cluster
 197047 – Failover/Failback policies on Microsoft Cluster Server
Windows 2003 Enterprise and DataCenter:

 301114 – Proper usage of the Preferred Owner List on a cluster Server (not
finished)
 299631 – Failover behaviour on clusters of three or more nodes
 296799 – How to configure Windows Clustering Groups for hot spare
support
22 Module 5: Clustering


Mount Point Drives


Mount Point Drives have been available since Windows 2000. They were,
however, not supported for use in Windows 2000 clusters. This essentially
limited the drive letter assignments in a Windows 2000 cluster to a maximum
of 26. This number included local drive assignments on each cluster node.
For example, a cluster node may have local physical disks A: (Floppy Drive),
C: (Local Disk) and D: (Local Disk). This would effectively leave 23 drives
that could be assigned to the cluster. As each node in the cluster may be a
possible owner of each of these 23 disk resources, this would limit the entire
cluster to 23 disk resources.
As the supported number of Nodes in clusters increases, this limitation begins
to seriously hamper the ability to build enterprise class solutions.
In Windows 2003 Server, support for mount point drives has been added in
Windows clusters.
Support for Mount Point Drives does not exist in Windows 2000.
A mount point drive is an entire volume that is mounted inside another folder
that is hosted by another volume.

Module 5: Clustering 23

The steps to create a mount point drive available for cluster use are as follows:

1. A new unformatted disk will be available in Disk Manager. Make sure that
it is a Basic Volume. Right-click it and choose new partition.

2. Choose Primary partition and click Next.