Exchange Solutions

Ran into a strange issue recently during an Exchange 2010 to 2016 migration. Internal mail sent from Exchange 2016 to Exchange 2010 was stuck in the mail queue. The queue viewer on Exchange 2016 reported the following error.

{LED=451 4.4.397 Error communicating with target host. -> 421 4.2.1 Unable to connect -> SocketTimedout: Socket error code 10060};{MSG=};{FQDN=<external.companyname.com>};{IP=<external IP>};

This is a fairly generic error and I have changed the FQDN and IP address in the example above. But the key here is that the Exchange 2016 server was trying to send all internal mail to the public IP of the Exchange 2010 server versus the internal IP.

For example, if a test user on Exchange 2016 tried to send an email to a test user on Exchange 2010, 2016 was routing the mail externally out of the firewall, only to try and hairpin back to one of the public-facing IPs.

This kind of hairpin attempted by Exchange was immediately blocked by the firewall which determined that internally sourced connections should not be trying to enter the public side of the firewall.

Ran into a strange issue recently where all the default address lists were missing in Exchange. Running a Get-AddressList in the Exchange Management Shell returned zero results. The address lists were also absent in the Exchange Admin Center as well.

When we attempted to recreate the missing address lists using the same name (for example, “All Users”) we received an error that the address list already existed.

 C:\> New-AddressList "All Users" -Included Recipients MailboxUsers
Active Directory operation failed on dc1.skaro.local. The object 'CN=All Users,CN=All Address Lists,CN=Address Lists Container,CN=SKARO,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=skaro,DC=local' already exists.
+CategoryInfo : Not specified (0:Int32) [New-AddressList], ADObjectAlreadyExistsException

If we attempted to modify this address list (or remove it) it reported it could not be found.

 C:\> Set-AddressList "All Users" -Included Recipients MailboxUsers
The operation couldn't be performed because object 'All Users' couldn't be found on 'dc1.skaro.local'.

Similarly, if we tried to create a brand new address list that we knew never existed in the environment previously this also failed.

 C:\> New-AddressList "Brand New List" -Included Recipients MailboxUsers
The operation couldn't be performed because object 'CN=Brand New List,CN=All Address Lists,CN=Address Lists Container,CN=SKARO,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=skaro,DC=local' couldn't be found on 'dc1.skaro.local'.

These address lists were also missing when we used ADSI Edit to examine the Address Lists Container.

I recently ran into an issue where the Microsoft Exchange FrontEnd Transport service refused to stay started. As a result, no external mail was being delivered to the Exchange server.

From the Services app, I could start the service, but it would stop within a couple of seconds.

When reviewing the Application logs in the Event Viewer I ran into a few separate errors with a source of MSExchangeFrontEndTransport. The first error was merely a symptom of a broken Front End Transport and not the root cause. We ignored this one and moved on.

Inbound direct trust authentication failed for certificate %1. The source IP address of the server that tried to authenticate to Microsoft Exchange is [%2]. Make sure EdgeSync is running properly.

The second error gave us the clue we needed. This error indicated something else was already listening to port 25.

The address is already in use. Binding: 0.0.0.0:25.

When installing an Exchange update you may run into the error; Service: MSExchangeHM failed to stop.

[ERROR] The following error was generated when "$error.Clear(); 
& $RoleBinPath\ServiceControl.ps1 -Operation:DisableServices -Roles
($RoleRoles.Replace('Role','').Split(',')) -SetupScriptsDirectory:$RoleBinPath;
& $RoleBinPath\ServiceControl.ps1 -Operation:Stop -Roles:
($RoleRoles.Replace('Role','').Split(',')) -IsDatacenter:([bool]$RoleIsDatacenter)" was run:
"Microsoft.Exchange.Configuration.Tasks.ServiceStopFailureException: 
 Service 'MSExchangeHM' failed to stop due to error:'Cannot stop MSExchangeHM service on computer '.'.'. 
---> System.InvalidOperationException:  Cannot stop MSExchangeHM service on computer  '.'. 
---> System.ComponentModel.Win32Exception:  The service has not been started   
    --- End of inner exception stack trace ---
    at System.ServiceProcess.ServiceController.Stop()

The MSExchangeHM service is part of the Managed Availability feature in Exchange. Managed Availability performs monitoring and self-healing to ensure Exchange stays up and running. This is a service you want to ensure stays running during normal operation, but in this particular scenario, we need to terminate the service so our cumulative update can complete.