Mark Minasi's Reader Forum
Mark Minasi's Reader Forum
Home | Profile | Register | Active Topics | Active Polls | Members | Search | FAQ | Minasi Forum RSS Feed
Username:
 Password:
Save Password
Forgot your Password?

 All Forums
 Email, Databases, Sharepoint and more
 Exchange
 Adding member to DAG "STILL" Fails		  New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 05/31/2012 :  11:41:30 AM  Show Profile  Reply with Quote
Test Setup.

Windows 2008R2 DC SP1 with Exchange Std 2010 SP2 and CAS and HTS Roles.

Windows 2008R2 SP1 with Exchange Ent 2010 SP2 - MBX 01 Server - Windows 2008R2 SP1 with Exchange Ent 2010 SP2 - MBX 02 Server

Mailflow works fine between both MBX Servers. Added one member to DAG works fine when try to add the second MBX02 Server I see the following in the log file....

Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering timed out while trying to validate server 'MBX02'. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server. ---

Got 2 NICS in each server. One for MAPI and other for Replication. Replication NICs have a direct CROSS OVER cable connection between them and no DNS or Gateway setup on it and Register in DNS Disabled and Netbios over TCP/IP Disabled.

Assigned a static IP to DAG and it shows up in the DNS and under the computer container on the AD.

I also tried disbaling the replicaiton NICs still no joy

Any ideas?

Advise please ...thanks!

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Edited by - MadCow on 06/25/2012 2:55:07 PM

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 05/31/2012 :  2:33:17 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
Please check if this applies: http://social.technet.microsoft.com/Forums/hu/exchange2010/thread/db14b6a4-844c-4df2-a0f8-d60ce3f945d2
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 05/31/2012 :  4:22:57 PM  Show Profile  Reply with Quote

Thanks Jazzy I tried that before but no joy. This is the second time I am stuck adding a second member node to a DAG. I re-did my entire test envoirenment.

And it looks like lots of other people meet this same issue.

I even disabled the IPv6 ia registry like you said.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/01/2012 :  11:45:06 AM  Show Profile  Reply with Quote

I am picking these errors on my both MBX nodes when I try to add the second node to the DAG.

Process STORE.EXE (PID=4784). All Domain Controller Servers in use are not responding: DC001.Exchange.LOCAL

DNS seems fine to me and name resolution/NSlookup all good. I ran flushdns on the DC/DNS Server and seems fine but still above errors. Its a small network with 1DC and 2 MBX Servers.

And I ran the DCDIAG /s:DCNAME and it also tells me the DNS is screwed but shows a wrong time.

The time on the MBX Servers and DC is in sync.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/01/2012 :  12:06:47 PM  Show Profile  Reply with Quote

Could this have something to do with the DC which is running off of a Virtual Host? Both MBX servers are phsyical.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Edited by - MadCow on 06/01/2012 2:00:40 PM
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/01/2012 :  3:09:46 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
Well, obviously something is wrong with communications between the servers. What was the exact error with DCDIAG and the time?
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/01/2012 :  3:51:35 PM  Show Profile  Reply with Quote

Directory Server Diagnosis


Performing initial setup:

* Identified AD Forest.
Done gathering initial info.


Doing initial required tests


Testing server: Default-First-Site-Name\DC001

Starting test: Connectivity

......................... DC001 passed test Connectivity



Doing primary tests


Testing server: Default-First-Site-Name\DC001

Starting test: Advertising

......................... DC001 passed test Advertising

Starting test: FrsEvent

......................... DC001 passed test FrsEvent

Starting test: DFSREvent

......................... DC001 passed test DFSREvent

Starting test: SysVolCheck

......................... DC001 passed test SysVolCheck

Starting test: KccEvent

......................... DC001 passed test KccEvent

Starting test: KnowsOfRoleHolders

......................... DC001 passed test KnowsOfRoleHolders

Starting test: MachineAccount

......................... DC001 passed test MachineAccount

Starting test: NCSecDesc

......................... DC001 passed test NCSecDesc

Starting test: NetLogons

......................... DC001 passed test NetLogons

Starting test: ObjectsReplicated

......................... DC001 passed test ObjectsReplicated

Starting test: Replications

......................... DC001 passed test Replications

Starting test: RidManager

......................... DC001 passed test RidManager

Starting test: Services

......................... DC001 passed test Services

Starting test: SystemLog

A warning event occurred. EventID: 0x00001695

Time Generated: 06/01/2012 12:12:26

Event String:

Dynamic registration or deletion of one or more DNS records associated with DNS domain 'Exchange.LOCAL.' failed. These records are used by other computers to locate this server as a domain controller (if the specified domain is an Active Directory domain) or as an LDAP server (if the specified domain is an application partition).


A warning event occurred. EventID: 0x00001695

Time Generated: 06/01/2012 12:12:27

Event String:

Dynamic registration or deletion of one or more DNS records associated with DNS domain 'DomainDnsZones.Exchange.LOCAL.' failed. These records are used by other computers to locate this server as a domain controller (if the specified domain is an Active Directory domain) or as an LDAP server (if the specified domain is an application partition).


A warning event occurred. EventID: 0x00001695

Time Generated: 06/01/2012 12:12:27

Event String:

Dynamic registration or deletion of one or more DNS records associated with DNS domain 'ForestDnsZones.Exchange.LOCAL.' failed. These records are used by other computers to locate this server as a domain controller (if the specified domain is an Active Directory domain) or as an LDAP server (if the specified domain is an application partition).


......................... DC001 passed test SystemLog

Starting test: VerifyReferences

......................... DC001 passed test VerifyReferences



Running partition tests on : ForestDnsZones

Starting test: CheckSDRefDom

......................... ForestDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... ForestDnsZones passed test

CrossRefValidation


Running partition tests on : DomainDnsZones

Starting test: CheckSDRefDom

......................... DomainDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... DomainDnsZones passed test

CrossRefValidation


Running partition tests on : Schema

Starting test: CheckSDRefDom

......................... Schema passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Schema passed test CrossRefValidation


Running partition tests on : Configuration

Starting test: CheckSDRefDom

......................... Configuration passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Configuration passed test CrossRefValidation


Running partition tests on : Exchange

Starting test: CheckSDRefDom

......................... Exchange passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Exchange passed test CrossRefValidation


Running enterprise tests on : Exchange.LOCAL

Starting test: LocatorCheck

......................... Exchange.LOCAL passed test LocatorCheck

Starting test: Intersite

......................... Exchange.LOCAL passed test Intersite

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/01/2012 :  3:55:53 PM  Show Profile  Reply with Quote

I can logon on MBX01 and add MBX02 to the DAG. But when I try add MBX01 to the DAG .... it falls flat.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/01/2012 :  4:21:08 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
Can you post the ipconfig /all of the three servers?
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

wobble_wobble
Honorable But Hopeless Addict

Ireland
4517 Posts
Status: offline
Posted - 06/02/2012 :  01:57:37 AM  Show Profile  Visit wobble_wobble's Homepage  Look at the Skype address for wobble_wobble  Reply with Quote
Add the cluster wizard to the 2 servers and see if it throws up the issue in validate cluster wizard.
Joe

After everything that has happened during the month of Jan 07, I do believe that pigs fly backwards!

http://whatismyv6.com/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  09:35:27 AM  Show Profile  Reply with Quote
IPCONFIG/All

Windows 2008R2 SP1 Domain Controller: Running CAS and HTS Roles - Exchange 2010 Std SP2


Windows IP Configuration

Host Name . . . . . . . . . . . . : DC001
Primary Dns Suffix . . . . . . . : Exchange.LOCAL
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : Exchange.LOCAL

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connection
Physical Address. . . . . . . . . : 00-50-56-96-60-9D
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 10.1.1.164(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.1.1.1
DNS Servers . . . . . . . . . . . : 10.1.1.164
NetBIOS over Tcpip. . . . . . . . : Enabled





Mailbox02 - Exchange 2010 Ent SP2



Windows IP Configuration

Host Name . . . . . . . . . . . . : MBX02
Primary Dns Suffix . . . . . . . : Exchange.LOCAL
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : Exchange.LOCAL

Ethernet adapter Local Area Connection* 9:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Failover Cluster Virtual Adapte
r
Physical Address. . . . . . . . . : EA-39-35-AA-F0-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter MBX02-MAPI:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : HP NC553i Dual Port FlexFabric 10Gb Conve
rged Network Adapter #2
Physical Address. . . . . . . . . : E8-39-35-AA-F0-E4
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 10.1.1.144(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.1.1.1
DNS Servers . . . . . . . . . . . : 10.1.1.164
NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter MBX02-Replication:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : HP NC553i Dual Port FlexFabric 10Gb Conve
rged Network Adapter
Physical Address. . . . . . . . . : E8-39-35-AA-F0-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 1.1.1.11(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
NetBIOS over Tcpip. . . . . . . . : Disabled


Mailbox01- Exchange 2010 Ent SP2




Windows IP Configuration

Host Name . . . . . . . . . . . . : MBX01
Primary Dns Suffix . . . . . . . : Exchange.LOCAL
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : Exchange.LOCAL

Ethernet adapter Local Area Connection* 9:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Failover Cluster Virtual Adapter
Physical Address. . . . . . . . . : 16-55-20-52-41-53
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter MBX01-MAPI:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : HP NC553i Dual Port FlexFabric 10Gb Converged Network Adapter
Physical Address. . . . . . . . . : E8-39-35-AB-C8-40
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 10.1.1.143(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.1.1.1
DNS Servers . . . . . . . . . . . : 10.1.1.164
NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter MBX01-Replication:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : HP NC553i Dual Port FlexFabric 10Gb Converged Network Adapter #2
Physical Address. . . . . . . . . : E8-39-35-AB-C8-44
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 1.1.1.10(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
NetBIOS over Tcpip. . . . . . . . : Disabled

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/04/2012 :  09:53:44 AM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
And can you post the relevant part from the Exchange Setup log too? I don't mind if it's a lot of text.
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  10:15:16 AM  Show Profile  Reply with Quote

Thanks Jazzy.

From MBX02 I successfully added MBX01 to the DAGroup and when I try to add MBX02 to the DAGroup here is the file ..... from today



add-databaseavailabiltygroupserver started on machine MBX02.
[2012-06-04T14:11:53] add-dagserver started
[2012-06-04T14:11:53] commandline: $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2012-06-04T14:11:53] Option 'Identity' = 'DAG001'.
[2012-06-04T14:11:53] Option 'MailboxServer' = 'MBX02'.
[2012-06-04T14:11:53] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2012-06-04T14:11:53] Option 'WhatIf' = ''.
[2012-06-04T14:11:53] Process: w3wp w3wp.exe:6172.
[2012-06-04T14:11:53] User context = 'NT AUTHORITY\SYSTEM'.
[2012-06-04T14:11:53] Member of group 'Everyone'.
[2012-06-04T14:11:53] Member of group 'BUILTIN\Users'.
[2012-06-04T14:11:53] Member of group 'NT AUTHORITY\SERVICE'.
[2012-06-04T14:11:53] Member of group 'CONSOLE LOGON'.
[2012-06-04T14:11:53] Member of group 'NT AUTHORITY\Authenticated Users'.
[2012-06-04T14:11:53] Member of group 'NT AUTHORITY\This Organization'.
[2012-06-04T14:11:53] Member of group 'BUILTIN\IIS_IUSRS'.
[2012-06-04T14:11:53] Member of group 'LOCAL'.
[2012-06-04T14:11:53] Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2012-06-04T14:11:53] Member of group 'BUILTIN\Administrators'.
[2012-06-04T14:11:53] Updated Progress 'Validating the parameters.' 2%.
[2012-06-04T14:11:53] Working
[2012-06-04T14:11:53] Mailbox server: value passed in = MBX02, mailboxServer.Name = MBX02, mailboxServer.Fqdn = MBX02.Exchange.LOCAL
[2012-06-04T14:11:53] LogClussvcState: clussvc is Stopped on MBX02.Exchange.LOCAL. Exception (if any) = none
[2012-06-04T14:11:53] The IP addresses for the DAG are (blank means DHCP): 10.1.1.96
[2012-06-04T14:11:53] Looking up IP addresses for DAG001.
[2012-06-04T14:11:53] DAG001 = [ 10.1.1.96 ].
[2012-06-04T14:11:53] Looking up IP addresses for mbx02.
[2012-06-04T14:11:53] mbx02 = [ 10.1.1.144, 1.1.1.11, ::1 ].
[2012-06-04T14:11:53] Looking up IP addresses for MBX02.Exchange.LOCAL.
[2012-06-04T14:11:53] MBX02.Exchange.LOCAL = [ 10.1.1.144, 1.1.1.11, ::1 ].
[2012-06-04T14:11:53] DAG DAG001 has 1 servers:
[2012-06-04T14:11:53] DAG DAG001 contains server MBX01.
[2012-06-04T14:11:53] Updated Progress 'Checking if Mailbox server 'MBX02' is in a database availability group.' 4%.
[2012-06-04T14:11:53] Working
[2012-06-04T14:11:53] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(MBX02.Exchange.LOCAL) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed... This is OK.
[2012-06-04T14:11:53] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2012-06-04T14:11:53] DumpClusterTopology: Opening remote cluster DAG001.
[2012-06-04T14:11:53] Dumping the cluster by connecting to: DAG001.
[2012-06-04T14:11:53] The cluster's name is: DAG001.
[2012-06-04T14:11:53] Groups
[2012-06-04T14:11:53] group: Available Storage [not a CMS]
[2012-06-04T14:11:53] OwnerNode: MBX01.Exchange.LOCAL
[2012-06-04T14:11:53] State: Offline
[2012-06-04T14:11:53] group: Cluster Group [Cluster Main Group]
[2012-06-04T14:11:53] OwnerNode: MBX01.Exchange.LOCAL
[2012-06-04T14:11:53] State: Online
[2012-06-04T14:11:53] Resource: Cluster Name [Online, type = Network Name, PossibleOwners = MBX01 ]
[2012-06-04T14:11:53] NetName = [DAG001]
[2012-06-04T14:11:53] Resource: Cluster IP Address [Online, type = IP Address, PossibleOwners = MBX01 ]
[2012-06-04T14:11:53] Address = [10.1.1.96]
[2012-06-04T14:11:53] EnableDhcp = [0]
[2012-06-04T14:11:53] Network = [Cluster Network 1]
[2012-06-04T14:11:53] Nodes
[2012-06-04T14:11:53] node: MBX01.Exchange.LOCAL [ state = Up ]
[2012-06-04T14:11:53] Subnets
[2012-06-04T14:11:53] Name(Cluster Network 1), Mask(10.1.1.0/24), Role(ClusterNetworkRoleInternalAndClient)
[2012-06-04T14:11:53] NIC 10.1.1.143 on Node MBX01 in State=Up
[2012-06-04T14:11:53] Name(Cluster Network 2), Mask(1.1.1.0/24), Role(ClusterNetworkRoleInternalUse)
[2012-06-04T14:11:53] NIC 1.1.1.10 on Node MBX01 in State=Up
[2012-06-04T14:11:53] Opening the cluster on nodes [mbx01].
[2012-06-04T14:11:53] Other mailbox servers in the DAG are already members of cluster 'DAG001'
[2012-06-04T14:11:53] The server MBX02 does not belong to a cluster, and the other servers belong to DAG001.
[2012-06-04T14:11:53] Successfully resolved the servers based on the stopped servers list.
[2012-06-04T14:11:53] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2012-06-04T14:11:53] The following servers are in the StoppedServers list:
[2012-06-04T14:11:53] Verifiying that the members of database availability group 'DAG001' are also members of the cluster.
[2012-06-04T14:11:53] Verifying that the members of cluster 'DAG001' are also members of the database availability group.
[2012-06-04T14:11:53] According to GetNodeClusterState(), the server MBX02 is NotConfigured.
[2012-06-04T14:11:53] The CNO is currently Online.
[2012-06-04T14:11:53] InternalValidate() done.
[2012-06-04T14:11:53] Updated Progress 'Adding server 'MBX02' to database availability group 'DAG001'.' 6%.
[2012-06-04T14:11:53] Working
[2012-06-04T14:11:53] Updated Progress 'Adding server 'MBX02' to the cluster.' 8%.
[2012-06-04T14:11:53] Working
[2012-06-04T14:11:54] The following log entry comes from a different process that's running on machine 'MBX01.Exchange.LOCAL'. BEGIN
[2012-06-04T14:11:54] [2012-06-04T14:11:54] Opening a local AmCluster handle.
[2012-06-04T14:11:54] Updated Progress 'Adding server 'mbx02' to database availability group 'DAG001'.' 2%.
[2012-06-04T14:11:54] Working
[2012-06-04T14:11:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = MBX02, dwStatus = 0x0 )
[2012-06-04T14:11:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseContinue, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 12, szObjectName = MBX02, dwStatus = 0x800713bb )
[2012-06-04T14:11:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 12, szObjectName = MBX02, dwStatus = 0x800713bb )
[2012-06-04T14:11:54] Found a matching exception: Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException: A server-side database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server 'MBX02'. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
[2012-06-04T14:11:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = MBX02, dwStatus = 0x0 )
[2012-06-04T14:11:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )

[2012-06-04T14:11:54] The preceding log entry comes from a different process running on computer 'MBX01.Exchange.LOCAL'. END
[2012-06-04T14:11:54] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2012-06-04_14-11-53.841_add-databaseavailabiltygroupserver.log".
[2012-06-04T14:11:54] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering timed out while trying to validate server 'MBX02'. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server. ---> Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException: A server-side database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server 'MBX02'. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
--- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException) ---
at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
at Microsoft.Exchange.Cluster.ReplayService.ReplayRpcServer.<>c__DisplayClass34.<RpcsAddNodeToCluster>b__33()
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
--- End of stack trace on server (MBX01.Exchange.LOCAL) ---
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperationDbName(AmServerName serverName, String databaseName, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperation(AmServerName serverName, Nullable`1 dbGuid, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2012-06-04T14:11:54] Updated Progress 'Done!' 100%.
[2012-06-04T14:11:54] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

aval
Honorable But Hopeless Addict

USA
3276 Posts
Status: online
Posted - 06/04/2012 :  10:20:33 AM  Show Profile  Reply with Quote
I noticed this in your DCDIAG output:

Dynamic registration or deletion of one or more DNS records associated with DNS domain 'Exchange.LOCAL.' failed. These records are used by other computers to locate this server as a domain controller (if the specified domain is an Active Directory domain) or as an LDAP server (if the specified domain is an application partition).

Does that error still appear if you run DCDIAG now?

Not sure if it has anything to do with the DAG problem but you can manually register these records as follows:

net stop netlogon
net start netlogon
Go to Top of Page

aval
Honorable But Hopeless Addict

USA
3276 Posts
Status: online
Posted - 06/04/2012 :  10:24:57 AM  Show Profile  Reply with Quote
And "ipconfig /registerdns" for the A record.
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  11:01:11 AM  Show Profile  Reply with Quote

Thanks Aval for your input. This is the result of DCDIAG /S:DC001 from today


Directory Server Diagnosis


Performing initial setup:

* Identified AD Forest.
Done gathering initial info.


Doing initial required tests


Testing server: Default-First-Site-Name\DC001

Starting test: Connectivity

......................... DC001 passed test Connectivity



Doing primary tests


Testing server: Default-First-Site-Name\DC001

Starting test: Advertising

......................... DC001 passed test Advertising

Starting test: FrsEvent

......................... DC001 passed test FrsEvent

Starting test: DFSREvent

......................... DC001 passed test DFSREvent

Starting test: SysVolCheck

......................... DC001 passed test SysVolCheck

Starting test: KccEvent

......................... DC001 passed test KccEvent

Starting test: KnowsOfRoleHolders

......................... DC001 passed test KnowsOfRoleHolders

Starting test: MachineAccount

......................... DC001 passed test MachineAccount

Starting test: NCSecDesc

......................... DC001 passed test NCSecDesc

Starting test: NetLogons

......................... DC001 passed test NetLogons

Starting test: ObjectsReplicated

......................... DC001 passed test ObjectsReplicated

Starting test: Replications

......................... DC001 passed test Replications

Starting test: RidManager

......................... DC001 passed test RidManager

Starting test: Services

......................... DC001 passed test Services

Starting test: SystemLog

......................... DC001 passed test SystemLog

Starting test: VerifyReferences

......................... DC001 passed test VerifyReferences



Running partition tests on : ForestDnsZones

Starting test: CheckSDRefDom

......................... ForestDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... ForestDnsZones passed test

CrossRefValidation


Running partition tests on : DomainDnsZones

Starting test: CheckSDRefDom

......................... DomainDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... DomainDnsZones passed test

CrossRefValidation


Running partition tests on : Schema

Starting test: CheckSDRefDom

......................... Schema passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Schema passed test CrossRefValidation


Running partition tests on : Configuration

Starting test: CheckSDRefDom

......................... Configuration passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Configuration passed test CrossRefValidation


Running partition tests on : Exchange

Starting test: CheckSDRefDom

......................... Exchange passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Exchange passed test CrossRefValidation


Running enterprise tests on : Exchange.LOCAL

Starting test: LocatorCheck

......................... Exchange.LOCAL passed test LocatorCheck

Starting test: Intersite

......................... Exchange.LOCAL passed test Intersite

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  11:06:34 AM  Show Profile  Reply with Quote

Thanks Joe. I tried to add the node via Failvoer Cluster Manager and got this ...

Node: mbx02.exchange.local
Started 6/4/2012 11:04:46 AM
Completed 6/4/2012 11:04:46 AM

Adding mbx02.exchange.local to the cluster.
Validating cluster state on node mbx02.
Unable to successfully cleanup.
The server 'mbx02.exchange.local' could not be added to the cluster.
An error occurred while adding node 'mbx02.exchange.local' to cluster 'DAG001'.
The cluster node is not reachable

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  11:21:05 AM  Show Profile  Reply with Quote
C:\>cluster.exe /cluster:DAG001 /add /node:MBX02.exchange.local

Ran the following command ...trying to add the MBX02 to the DAGroup ...

Configuring node MBX02.exchange.local
---------------------------------------
12% Validating cluster state on node MBX02.This phase has failed for Cluster object 'MBX02' with an error status of -2147019845 (0x800713BB).

This phase has failed for Cluster object 'MBX02' with an error status of -2147019845 (0x800713BB).
Cleaning up MBX02.

System error 5051 has occurred (0x000013bb).
The cluster node is not reachable.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/04/2012 :  11:41:55 AM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
Can you check the DAG computer account with ADISedit or Atrribute Editor in ADUC? Is the DNSHostname atrribute there?
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  12:18:41 PM  Show Profile  Reply with Quote

Thanks jazzy.

Yes, the attrib is present.

What you think about this article ....

http://blogs.technet.com/b/roplatforms/archive/2010/04/28/the-case-of-the-server-who-couldn-t-join-a-cluster-operation-returned-because-the-timeout-period-expired.aspx

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  12:31:36 PM  Show Profile  Reply with Quote
I am seeing the following in the
C:\Windows\Cluster\Reports\ValidateStorage file

10.1.1.143 = MBX01 (Rep NIC: 1.1.1.10)
10.1.1.144 = MBX02 (Rep NIC: 1.1.1.11)


CprepDiskGetProps: Failed to get adapter descriptor for disk 1, status 1168. Continuing...

0000250c.000026a4::15:52:49.433 DoIoctlAndAlloc: ControlCode 0x70050, retCode 1, status 122, buffer size 800

0000250c.000026a4::15:52:49.448 DoIoctlAndAlloc: ControlCode 0x70050, retCode 1, status 122, buffer size 800

0000250c.000026a4::15:52:49.480 IsDynamicDisk: Exit IsDynamicDisk: DynamicDisk 0, status 0

0000250c.000026a4::15:52:49.480 CprepDiskGetProps: Exit CprepDiskGetProps: hr 0x0, DiskProps->Flags 0xa300

0000250c.000026a4::15:53:33.051 Loading iphlpapi.dll for ICMP echo routines ...
0000250c.000026a4::15:53:33.144 GetIpConfigSerialized
0000250c.000026a4::15:53:33.410 GetIpConfigSerialized
0000250c.000026a4::15:53:34.268 Sending ICMP echo packet from 10.1.1.143 to 10.1.1.144

0000250c.000026a4::15:53:34.299 Elapsed time = 0

0000250c.000026a4::15:53:34.346 Sending ICMP echo packet from 1.1.1.10 to 1.1.1.11

0000250c.000026a4::15:53:34.377 Elapsed time = 0

0000250c.000026a4::15:53:40.476 Sending ICMP echo packet from 10.1.1.143 to 1.1.1.11

0000250c.000026a4::15:53:42.364 Icmp Echo failed with error 80072b02 (11010).

0000250c.000026a4::15:53:42.411 Sending ICMP echo packet from 10.1.1.143 to 1.1.1.11

0000250c.000026a4::15:53:44.361 Icmp Echo failed with error 80072b02 (11010).

0000250c.000026a4::15:53:44.392 Sending ICMP echo packet from 10.1.1.143 to 1.1.1.11

0000250c.000026a4::15:53:46.358 Icmp Echo failed with error 80072b02 (11010).

0000250c.000026a4::15:53:46.498 Sending ICMP echo packet from 1.1.1.10 to 10.1.1.144

0000250c.000026a4::15:53:46.529 Icmp Echo failed with error 800704cf (1231).

0000250c.000026a4::15:53:46.545 Sending ICMP echo packet from 1.1.1.10 to 10.1.1.144

0000250c.000026a4::15:53:46.560 Icmp Echo failed with error 800704cf (1231).

0000250c.000026a4::15:53:46.592 Sending ICMP echo packet from 1.1.1.10 to 10.1.1.144

0000250c.000026a4::15:53:46.623 Icmp Echo failed with error 800704cf (1231).

0000250c.000026a4::15:53:56.825 FinalRelease: Enter: FinalRelease:



***** Validate Server Stop ****

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Edited by - MadCow on 06/04/2012 12:33:07 PM
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/04/2012 :  12:56:36 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
quote:
Originally posted by MadCow


Thanks jazzy.

Yes, the attrib is present.

What you think about this article ....

http://blogs.technet.com/b/roplatforms/archive/2010/04/28/the-case-of-the-server-who-couldn-t-join-a-cluster-operation-returned-because-the-timeout-period-expired.aspx

Well, is the DHCP client running on you servers? Enabled IPv6 again? (use the fixit in http://support.microsoft.com/kb/929852)

I was reading through this http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/71960757-66fc-4aea-81ba-3783a48401a0/ and suspecting your domain name to be a problem. I check with Microsoft but according to them there's no such issue known, they say it seems to be a network or DNS problem.
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/04/2012 :  2:53:56 PM  Show Profile  Reply with Quote

Thanks Jazzy much.

I will re-do my test infrastructure with a different domain name.

I WILL BE BACK!

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/25/2012 :  2:45:32 PM  Show Profile  Reply with Quote
Changed the Domain name and restarted all over but still the same issue. Able to add first member to the DAG when trying to add the second ... I see this in the application log ....


DCOM was unable to communicate with the computer MBX002 using any of the configured protocols.

The exchangesetup logs she this ...


A server-side database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server 'MBX002'. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.

Firewalls are disable on all the servers and no network delays or issues.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Edited by - MadCow on 06/25/2012 2:54:40 PM
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/25/2012 :  3:04:36 PM  Show Profile  Reply with Quote

Ran the Validate this Cluster Configuration Wizard and the results
say ... No disks were found on which to perform cluster validation tests.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 06/25/2012 :  3:22:24 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
quote:
Originally posted by MadCow


Ran the Validate this Cluster Configuration Wizard and the results
say ... No disks were found on which to perform cluster validation tests.

That's expected, you can uncheck Disks when validating a cluster for Exchange (no shared disks).

Did you consider opening a case with Microsoft? Costs around 300 euro but may be money well spend in this case, considering the time you already spent on the issue.
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 06/25/2012 :  3:25:38 PM  Show Profile  Reply with Quote

Thanks Jazzy.

This is my 3rd attemplt in a test envoirenment.

Exactly this is what I was discussing with my manager. That I prepare a production evoirenment and if still does not work then we contact the mothership.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

MadCow
Honorable But Hopeless Addict

Canada
1834 Posts
Status: offline
Posted - 07/05/2012 :  6:37:16 PM  Show Profile  Reply with Quote

Today I successfully deployed 2 members in the DAG Nodes and no issues what so ever and this is prodcution

Failed 3 times in test envoirenment.

Though from time to time I do notice slight delay in replication.


Thank Jazzy.

Sunny
__________________________________________________________________________


"Everyone is susceptible to the notion that when you begin to do well, you begin to see no boundary lines and forget the rules apply" - Eliot Spitzer
Go to Top of Page

Jazzy
Administrator

Netherlands
1932 Posts
Status: offline
Posted - 07/05/2012 :  6:45:42 PM  Show Profile  Visit Jazzy's Homepage  Click to see Jazzy's MSN Messenger address  Reply with Quote
Usually it's the other way around. Everyting works fine in the lab but not when you deploy it in production.

Anyway, glad this is working now.
Jetze Mellema

Exchange specialist
Former MVP (2005-2012)
My blog: http://jetzemellema.blogspot.com (Dutch)
My company: http://www.imara-ict.nl/
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
Mark Minasi's Reader Forum © 2002-2011 Mark Minasi Go To Top Of Page
This page was generated in 0.33 seconds. Snitz Forums 2000