Something About Rolling Out Kerberos ASA Credentials Between Exchange 2013 and 2016 Servers

Hi folks,

As you well know that Exchange 2013 and 2016 servers can co-exist with each other well behind the same load balancer. Furthermore, they can also share the same ASA credentials for Kerberos authentication. And this article speaks about it.

However, when it came to roll out credentials from Exchange 2013 CAS server to Exchange 2016 Mailbox server (yes, yes, this is not a mistake, as Exchange 2016 has only Mailbox role with CAS services integrated) I could not run RollAlternateServiceAccountPassword.ps1 to deploy credentials from one of my current Exchange 2013 CAS servers to Exchange 2016 Mailbox server.

When running it from Exchange 2013 server against Exchange 2016 server, I would encounter this error:
RecordErrors : Cannot convert value "SERVER01" to type "Microsoft.Exchange.Data.Directory.Management.ClientAccessServer"

This is because I tried to run from the older version of the server on the new one.

Then I attempted to run the same command on the Exchange 2016 server. It produced this error:

Now the trick is that CAS role doesn't exist any longer with Exchange 2016 and instead of *-ClientAccessServer cmdlets Microsoft has introduced *-ClientAccessService cmdlets to manage CAS services on Exchange 2016 Mailbox servers. However these cmdlets are not supported on Exchange 2013 and therefore when script attempts to remotely talk to an Exchange 2013 box and retrieve ASA creds from there, it fails because Get-ClientAccessService command is simply not supported by Exchange 2013. However, since *-ClientAccessServer commands are still supported (even though they warn you about future deprecation) they work just as on Exchange 2013. So, on my first Exchange 2016 server I have edited RollAlternateServiceAccountPassword.ps1 and replaced all the -ClientAccessService entries with -ClientAccessServer.

After this I have kicked off again:

RollAlternateServiceAccountPassword.ps1 -ToSpecificServers SERVER02.contoso.com -CopyFrom SERVER02.contoso.com

This time, despite cmdlet deprecation warning copy of credentials succeeded which I can clearly see by executing the following command:

Get-ClientAccessService Server01 -IncludeAlternateServiceAccountCredentialPassword |fl Na
me, AlternateServiceAccountConfiguration

Finally, after ASA creds have been deployed to the first Exchange 2016 server I can go to other Exchange 2016 boxes and copy credentials from the first server without editing RollAlternateServiceAccountPassword.ps1 and any error because now creds were copied from Exchange 2016 to Exchange 2016.

I hope you will find this article useful for your troubleshooting and configuration.

And a little bonus, if for some reason you will need to remove ASA creds from your server you will need one of these commands:

For Exchange 2010 and 2013:

Set-ClientAccessServer CAS-1 -RemoveAlternateServiceAccountCredentials

For Exchange 2016 and probably later versions:

Set-ClientAccessService CAS-1 -RemoveAlternateServiceAccountCredentials

Enjoy!

View Schema Version PowerShell

Hi folks,

I have recently discovered a nice way to retrieve schema version using PowerShell. Earlier I published how to do it using dsquery command. However with dsquery you need to remember the DN for your forest. This one-liner makes life a lot easier:

Get-ADObject (get-adrootdse).schemaNamingContext -Property objectVersion

About each version of Windows Server in schema you can read here.

Enjoy!

Deleting Faulty Default Mailbox Databases

Hi folks,

Just wanted to share with you about the adventure I've had recently. As you know when Exchange Server 2010 and later is installed on the mailbox servers default mailbox database is created named something like "Mailbox Database 123456". Normally, new highly available databases are created which are stored outside of the default installation drive. Normally they are deleted.

When I was trying to delete my database I would see error as below:

I would see the same when trying to dismount database (which is also an invisible step when a database is being deleted):

As this article suggests this database has misconfigured MsExchMasterServerOrAvailabilityGroup property. When a mailbox server is a member of DAG this value should be equal to DN of DAG, for a standalone mailbox server this value will be DN of server.

When running Get-MailboxDatabase  DB01 | Format-Table Name, MasterType against my problematic database, I would see value of the MasterType property (corresponds to the MsExchMasterServerOrAvailabilityGroup attribute mentioned above) as Server. It indicates that DAG doesn't own this DB. Normally when a server is added to DAG MasterType value should be DatabaseAvailabilityGroup. So in my case this value wasn't properly updated and I was getting the error I was getting when attempting to delete or dismount DB.

The only way to fix it is tweaking Exchange objects in the AD on low level. Make sure it is backed up because AdsiEdit is not nice enough to warn you when you mistakenly delete something or mess a setting up. Therefore you should really know what you do. Or have MS support being at call with you and direct you as you tweak AD objects.

To access DB properties we will need to ADSIEDIT.MSC and navigate to CN=Databases,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=CONTOSO,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com .

This is the AD container where Exchange stores information about mailbox databases. If mailbox database us called DB01 you will need to edit properties of CN=DB01 subcontainer.

We will need to open properties of mailbox database in AD and check the value of the attribute. In my case it was distinguished name (DN) of the mailbox server where it was hosted: CN=SERVER01,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=CONTOSO,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com

I had to tweak this value to the DN of DAG my server is part of:

CN=DAG01,CN=Database Availability Groups,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=CONTOSO,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com

After this I gave some time for the AD to replicate. After which I have executed the following command again:

Get-MailboxDatabase  DB01 | Format-Table Name, MasterType

This time output was like below:

After which you can run the following command to remove mailbox database:
Remove-MailboxDatabase DB01 -Confirm:$false

After hitting Enter the default database should be successfully removed. At least it was a case for me.

Enjoy!

Which Domain Controller Can I Use for My EMS Commands

Hi folks,

I have previosly discussed how to use PowerShell commands (from the AD module) to discover domain controllers in an AD site. You can read about it in this post.

The same is possible to do by using EMS. In some of the cases when you are running multiple commands and dependent on the AD replication before running the next command it is nice to write all changes to the same AD command. MS has nicely provided -DomainController parameter which allows us to perform any reading/writing of Exchange information against the same domain controller. All you need is to specify DC host name.

Now if you have multiple domains or a lot of domain controllers within AD site where your Exchange server is located (or workstation with Exchange admin tools) you can use the Get-DomainController cmdlet to discover them. You can read about this cmdlet and all the available parameters here. Let's imagine that my Exchange server is located in the AD site called New York and in multi-domain forest. I need to write to a DC in the contoso.com domain. Then the following command will be used:

Get-DomainController -DomainName contoso.com |Where-Object {$_.AdSite -like "*NewYork*"} |select DnsHostName

It is pretty obvious. However, let me explain why I used filter for AdSite parameter and asked command to output DnsHostName parameter. Usually Get-DomainController returns distinguished value for AD site something like contoso.com/Configuration/Sites/New York. The same applies for domain controllers' parameter called Identity parameter of domain controller. It will be something like contoso.com/Configuration/Sites/New York/NYDC01. DnsHostName parameter will retrieve FQDN of DC which you can later use as value for the -DomainController parameter, like NYDC01.contoso.com.

I hope you will find this useful for your PowerShell adventures.

Enjoy!

Dealing With Failed Passive Copies

Hi folks,

I would love to share with you about my recent experience which I had with mailbox DB copies. I had an issue where mailbox DBs on one of the servers in DAG which contains only passive DB copies only some of the copies couldn't replicate if they got activated on one of the servers in primary datacenter. I also saw the following event in the Application log:

Some investigations lead me to find that this server was missing static route for the replication link.

It was fairly easy to resolve as I had to add persistent static route for replication network by running route add -p command with the appropriate information for route. After adding route replication was sorted out.

Now I had to sort out the mailbox copies themselves. I used the Get-MailboxDatabaseCopyStatus command on the server where these copies were failed which without any specified parameters retrieves mailbox databases copies hosted on the server from which you run it explicitly. Since all the copies on my servers were only passive I could filter them out by running the following command:

Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}

It retrieves all the copies DB copies that don't have status of healty. If you also have active DB copies you may also get all the mounted DB copies as their status will be Mounted. In this case you will need to tweak command to cater for these DB copies as well. So after I restored replication I ran this command to first suspend and then to resume copies:

Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}| foreach {Suspend-MailboxDatabaseCopy -Identity $_.Name -Confirm:$false}

However, after trying to suspend I got an error as below because of double pipelines:

So instead I dumped all the failed copies into a variable called $FailedCopies. My code turned out something as below:

$FailedCopy = Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}

After this I have ran commands to suspend and resume database copies against all DBs that were in variable:

$FailedCopy |foreach {Suspend-MailboxDatabaseCopy -Identity $_.Name -Confirm:$false}
$FailedCopy |foreach {Resume-MailboxDatabaseCopy -Identity $_.Name}

Finally if you decide to reseed (it may and will consume a lot of network traffic) you may run command as below:

$FailedCopy |foreach {Update-MailboxDatabaseCopy -Identity $_.Name -DeleteExistingFiles}

Enjoy!

Creating GPO Report Using PowerShell

Hi folks,

Just wanted to share with you about a nice method for creating HTML reports for more than one server by using PowerShell. In pre-Windows 2012 environment there was a nice command gpresult /h filename.htm. One drawback of it that it is local.

I recently needed to generate GPO report on multiple Exchange server. This is where the Get-GPResultantSetOfPolicy cmdlet came to help. You can read about this cmdlet here.

The below command ran Get-GPResultantSetOfPolicy against all Exchange servers that were piped by using Get-ExchangeServer command and for each of them generates a report that contains server name:

Get-ExchangeServer EXCH* |foreach {Get-GPResultantSetOfPolicy -computer $_.Name -reporttype html -path D:\InfoSec\Reports\$_.Name.html}

I hope you will find it useful.

Enjoy!

Some Adventures with Exchange Monitoring

Hi folks,

Just wanted to share with you about my recent adventure. As you well know when you are installing CU on Exchnage 2013/16 server you will need to place this server on maintenance mode before the installation and resume it back to production afterwards. I have already published here about a nicer way of doing it.

Recently I have performed installation of CU on my estate and when running scripts that restores my server to a production mode I noticed that some of the components, namely CAS front end components ended up in the Inactive state.

No matter what I did they failed to activate.

This, this and also this blog posts became my starting points. However running commands against each of Inactive components would not help whatsoever. However what I noticed that the free disk space on the C drive was less than 10%. It was about 10 GB which was enough for Windows OS to run but not to Exchange. I have identified a lot of rubbish on my C drive that occupied its valuable space and deleted it. Thus I have recovered more than 50% of free space. After which running below commands against each of server components succeeded:

Set-ServerComponentState SERVER01 -Component AutoDiscoverProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component ActiveSyncProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component EcpProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component EwsProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component OwaProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component OabProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component RpsProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component RwsProxy -State Active -Requester HealthAPI
Set-ServerComponentState SERVER01 -Component PRCProxy -State Active -Requester HealthAPI

This drew me to conclusion that this was one of the protective measures of Exchange server monitoring component which runs on backgrounds to ensure that all services are healthy and tries to automatically fix them, It simply wouldn't allow server to start when free disk space on system drive was below a certain threshold. I may need to read more about it as this component is quite mysterious and nothing much you can read about it or do about it, except some overrides.

Alternatively you can reactivate all components on server by running the below 2 commands:

Set-ServerComponentState SERVER01 -Component ServerWideOffline -State Inactive -Requester HealthAPI

Set-ServerComponentState SERVER01 -Component ServerWideOffline -State Active-Requester HealthAPI

After which you should get something as below:

Enjoy!

Seeing Exchange 2010 Databases in Exchange 2013/16 EMS

Hi folks,

Just wanted to share with you about quick way to deal with Exchange 2010 databases in Exchange 2013/16 management shell. Of course you should not try to change their settings via the later version EMS, as it's MS's golden rule that you manage Exchange with its own tools. However you may need this command to retrieve some of the settings from your 2010 databases in order to configure the same for your 2013/16 databases. Or retrieve mailboxes in these DB for further move to the latter version.

The command below will allow you to retrieve Exchange 2010 DBs info from the Exchange 2013/16 MS:

Get-MailboxDatabase -Identity DB00* -IncludePreExchange2013

Above sample retrieves all Exchange 2010 mailbox databases that start with DB00. If you don't include IncludePreExchange2013 you will get Exchange 2013/16 databases only or your AD will complain that it doesn't have this information.

Enjoy!

Another Way of Managing Page File From PowerShell

Hi folks,

In one of my previous posts I shared with you how to use PowerShell to configure virtual memory on your server.

However lately I have found another nicer way of managing paging file on your server by means of this Page File Module which you can download from MS TechNet gallery.

Before using commands from page file management module you will need to unzip it to a folder, let's call it C:\PowerShell\PageFile. After this you will need to load this module to your PowerShell session as below:

cd C:\PowerShell\PageFile
Import-Module .\AdjustVirtualMemoryPagingFileSize\AdjustVirtualMemoryPagingFileSize.psm1

This module has the following 2 commands:

You will need the Set-OSCVirtualMemory cmdlet that is the part of this module to configure size and location of the paging file.

To configure paging file of the size of 16GB+10MB on the drive C you will need to run the following command:

Set-OSCVirtualMemory -InitialSize 16394 -MaximumSize 16394 -DriveLetter "C:"

Enjoy!

Exchange CU Installation and NetBackup

Hi folks,

I want to share with you quickly about another error you may face when upgrading your Exchange 2013/16 server with Cumulative Updates. In my case it was that setup stopped with this error:

Setup can't continue with the upgrade because the monad (7452), monad (13944), monad (15320), monad (15356), monad (15416) has open files. Close the process, and then restart Setup

Of course process ID for monad on your server may be different. Investigation led me to this TechNet forum thread. It was pointing that Symantec NetBackup has been holding some of the files open which prevented successful proceeding with the installation. To solve this I have simply stopped all the NetBackup related services (of course you will need to negotiate it with your backup team) and restarted the installation process which completed successfully.

Enjoy!

PowerShell Module for Managing Registry Remotely

Hi folks,

I have recently had an experience where I needed to change registry on more than one server. And of course after this change I wanted to be sure that the registry change has been successful. Of course this can be done by logging in to each of these servers, but I wanted a simple and nice way of reporting of this registry change.

My investigations led me to this site from where you can download remote registry management PowerShell module. From this page you can download this module in form of a ZIP file. When extracted, you will end up having the PSRemoteRegistry folder which contains all the files necessary for module. You will need to copy the whole folder (not just contents) to %WINDIR%\System32\WindowsPowerShell\v1.0\Modules.

As soon as it the folder has been copied you can launch module in PowerShell session:

Import-Module PSRemoteRegistry

This module includes not only commands for reporting registry keys and their values but also for creating and tweaking keys. For the sake of my work I needed only Get-RegKey command which would talk to all my Exchange servers and retrieve the registry key which I had just created along with its configured values and drop them nicely to a report that can be read in Excel.

Imagine a scenario that you need to migrate IRM from one AD RMS cluster to another. For users to access and process emails sent by the previous RMS cluster you will need to establish redirection of old AD RMS cluster URL to the new one. This is done by tweaking registry and fully described here.

I used good old New-Item and New-ItemProperty commands to create registry keys and properties. After they have been created I executed the following code against my all Exchange 2013 and 2016 servers from EMS:

Get-ExchangeServer -Identity $_.ServerName |where {$_.AdminDisplayVersion -like "Version 15*"} |Get-RegKey -ComputerName $_.Name -Key SOFTWARE\Microsoft\ExchangeServer\v15\IRM -Name LicenseServerRedirection |Get-RegValue |select ComputerName,Key,Value,Data,Type |Export-Csv D:\scripts\RMS\Redirection.csv

It first found my all Exchange 2013/16 servers in AD and then executed Get-Reg key against all of them along with the values that i needed to retrieve. The output was nicely placed to CSV file which I can now easily access and read.

Enjoy!

Preparing ReFS Volumes for Exchange 2016 DAG

Hi folks,

According to the Exchange 2016 Preferred Architecture by Microsoft disk drives should be formatted  with ReFS with the integrity feature disabled. This file system is better for larger volumes of data and also has a lot of improvemets over NTFS on how it manages data. You can read more of it here.

Unfortunately formatting drives via GUI is not available in the Disk Manager GUI tool. And this is where PowerShell comes to our help.

Below I will share with you the commands I used to create volume mount points and format them with ReFS. This is the whole process of building DAG with AutoReseed.

First we will need to prepare directories structure for volume mount points that will be used by AutoReseed for volumes and for databases:

New-Item C:\Volumes -Type Directory
New-Item C:\Databases -Type Directory
New-Item C:\Volumes\Vol01 -Type Directory
New-Item C:\Volumes\Vol02 -Type Directory
New-Item C:\Volumes\Vol03 -Type Directory
New-Item C:\Databases\DB01 -Type Directory
New-Item C:\Databases\DB02 -Type Directory

Then we will create partitions. Against all volumes, including the spare one, you will need to run this command to create partition that uses the whole space:

New-Partition -DiskNumber X -UseMaximumSize

After partitions are created we will need to format volumes with ReFS with disable integration streams. This is not possible with the Disk Manager MMC console and therefore PowerShell is the best way to achieve it;

Get-Partition –Disknumber X –PartitionNumber 2 | Format-Volume –FileSystem ReFS –NewFileSystemLabel "ExchVol00$VolNum" -AllocationUnitSize 65536 -SetIntegrityStreams:$false –Confirm:$false

After volumes have been formatted you will need to add volume mount points to volume directory and directory that corresponds to each database that will be stored on this volume like below:

Add-PartitionAccessPath -DiskNumber X -PartitionNumber 2 -AccessPath C:\Volumes\Vol01
Add-PartitionAccessPath -DiskNumber X -PartitionNumber 2 -AccessPath C:\Databases\DB01
Add-PartitionAccessPath -DiskNumber X -PartitionNumber 2 -AccessPath C:\Databases\DB02

Don't also forget to configure spare volume without any mount points to databases:

Add-PartitionAccessPath -DiskNumber X -PartitionNumber 2 -AccessPath C:\Volumes\Vol03

I hope you will find it helpful for a small Exchange deployments, however for large Exchange deployments I would recommend you using scripts that are generated in the Exchange capacity calculator. It can be found on the Distribution tab of the calculator. All the buttons are self explanatory so I won't explain all of them here.

Enjoy!

Convert IP-less DAG to IP-based DAG

Hi folks,

As you well know Exchange 2013 SP1 and later installed on Windows Server 2012 R2 supports DAG without IP address. You can read more about it here and here. It is much easier to configure as there is no dependency on the availability of the IP address and CNO object in the AD. According to Microsoft it also reduces surface of attack on the server.

One important notice for IP-less DAGs is that you should well plan it. Make sure that application that integrate with Exchange 2013 or 2016 can work with IP-less DAG. The problem is that you won't be able to convert from IP-less DAG to IP-based DAG on fly and vice versa. In order to convert DAG from IP-less to IP-based you will need to destroy IP-less DAG and create a new IP-based DAG.

You can still convert DAG from IP-less to IP-based while preserving user data. First you will need to activate all DBs in DAG on one of the Exchange servers by running the command as below:

Get-MailboxDatabase DB001 |Move-ActiveMailboxDatabase -ActivateOnServer SERVER01

After this you will need to remove all passive copies from other servers:

Remove-MailboxDatabaseCopy DB001\SERVER02 -Confirm:$false

If your DAG runs in Datacenter Activation Coordination mode (as any post-Exchange 2010 SP1):

Set-DatabaseAvailabilityGroup -Identity DAG01 -DatacenterActivationMode Off

After DAC mode is disabled you need to remove all servers from DAG by running the below command against each server:

Remove-DatabaseAvailabilityGroupServer -Identity DAG01 -MailboxServer SERVER02 -confirm:$false

After none of the servers left in the DAG you can kill it:

Remove-DatabaseAvailabilityGroup -Identity DAG01 -confirm:$false

Of course you will be building your Exchange 2013 or 2016 DAG on Windows 2012 R2 servers. To succeed you will need to prestage CNO for your DAG in  AD based on the instruction from this TechNet article. If this step has not been performed attempts to add mailbox servers to DAG will fail with the following error:

Please retry the operation. Error: The fully qualified domain name for node ‘XXXX’ could not be found.

When everything is ready you can create new DAG with the IP address by running a command similar to this one:

New-DatabaseAvailabilityGroup -Name DAG01 -WitnessServer FSW01.contoso.com -WitnessDirectory C:\Witness\DAG01 -DatabaseAvailabilityGroupIPAddresses 10.100.0.160, 10.100.10.160

Then you can add all of your servers to DAG by running the following commands against each of them:

Add-DatabaseAvailabilityGroupServer -Identity DAG01 -MailboxServer SERVER01 -Verbose

After all servers are added to DAG you will need to enable DAC mode:

Set-DatabaseAvailabilityGroup -Identity DAG01 -DatacenterActivationMode DagOnly

If you use AutoReseed you will need to configure DAG with the parameters needed for AutoReseed and number of DB copies per volume:

Set-DatabaseAvailabilityGroup DAG01 -AutoDagVolumesRootFolderPath "E:\ExchVols"
Set-DatabaseAvailabilityGroup DAG01 -AutoDagDatabasesRootFolderPath "E:\ExchDB"
Set-DatabaseAvailabilityGroup DAG01 -AutoDagDatabaseCopiesPerVolume 2

I have already posted about AutoReseed configuration and operations previously.

When DAG is ready you can add mailbox database copies to the remaining servers:

Add-MailboxDatabaseCopy  -Identity "DB001" -MailboxServer "SERVER02" -ActivationPreference 2
Add-MailboxDatabaseCopy  -Identity "DB001" -MailboxServer "SERVER03" -ActivationPreference 3

Finally, when your DAG is ready and all copies are healthy you can redistribute them based on the activation preferences so that the load spread evenly across your Exchange servers:

RedistributeActiveDatabases.ps1 -DagName DAG01 -BalanceDbsByActivationPreference -Confirm:$False

So your DAG has been rebuilt and data preserved.

Enjoy!

Interactive Script to Configure More than One Static Global Catalog

Hi folks,

Just wanted to shared with you a script that will allow you to configured static global catalogs for any Exchange 2010 and later environment. Of course this is not a best practice to hard code global catalogs on Exchange servers according to Microsoft, however reasons may arise why you want to limit your Exchange servers to talk to this or that GC only. Mostly this is due to the performance of GCs which presents risk that an Exchange server can pick up a GC that is low on resources.. When putting together a script I assumed that you want to make static GCs resilient and therefore to configure your server to use 2 GCs. If you want to use more GCs you will need to add variable $GC3. If you want to reduce number of GCs to 1 then you will need to remove all references to $GC2.

The value for each $GCx variable should be a FQDN of a GC.

The code will look like something below:


$GC1 = Read-Host "Type names of Global Catalogs to be used as static GC for exchange. F.e Server1.contoso.net"
$GC2 += Read-Host "If more than one static Glbal Catalog is in use type its name. F.e Server2.contoso.net"
$Server = $(Get-WmiObject Win32_Computersystem).name
Set-ExchangeServer $Server -StaticGlobalCatalogs $GC1,$GC2

Enjoy!

UM Language Pack Adventures

Hi folks,

I would love to share with you about my recent adventure. I had to install 3 Exchange 2010 UM servers. Installation appeared to finish fine. However when I started installation of UM language packs I was presented with the following error:

As troubleshooting I started checking Exchange installation log that is created during installation (ExchangeSetup) and I found the following entry:

Further digging in the ExchangeSetup.log lead me to the following error entries:

[08/11/2016 14:31:36.0326] [2] [ERROR] Unexpected Error
[08/11/2016 14:31:36.0326] [2] [ERROR] Service 'MSExchangeADTopology' failed to reach status 'Running' on this server.

So it appears that AD topology service failed to start at the installation and therefore failed to register UM role in the AD. Interestingly enough all the UM-related services were running.

To resolve this problem I re-ran Exchange 2010 installation program.

After instillation successfully completed I kicked off installation of language pack. This time I wasn't faced by any errors and was able to proceed with the installation.

When checking installation log I saw that setup program was able to read UM role from the AD.

Finally, when checking registry (HKLM\Software\Microsoft\ExchangeServer\v14\UnifiedMessagingRole\LanguagePacks) for language pack I was able to see that in addition to the default en-US my language pack was also installed):

I hope you will find it useful.

Enjoy!

Automatic Uninstall of Windows Management Framework

Hi folks,

I would love to share with you about an experience I have recently had. I was working on the installation of Exchange 2016 servers on servers running Windows Server 2012 R2. The standard image that I had came with the PowerShell 5.0 with which Exchange 2016 can't work. PowerShell 5.0 comes for Windows 2012 R2 as part of the Windows Management Framework 5.0.

So before installing Exchange 2016 you will need to uninstall it to avoid issues when running EMS. It can be done easily from Windows Control Panel. However there is also an automatic way of uninstalling it.

This method can be useful when you are trying to uninstall it from more than one server or when you are trying to understand any other Windows updates or hotfixes. In the Technet gallery there is a module that allows to perform installation. It can be located here.

After module is downloaded you will need to load it to your PowerShell section and then you will need to execute the Uninstall-OSCHotfix cmdlet with the number of the KB of the update you want to uninstall. In case of the Windows Management Framework it is KB3134758.

In my case script the code looked like something as below. Please note that the path to the PSM1 file may vary.

Import-Module c:\Scripts\UninstallHotFixPowerShell\UninstallHotFix.psm1
Uninstall-OSCHotfix -HotFixID KB3134758

It will look something like this:

After update is uninstalled you will need tor reboot your server for it to take effect. After which you can confirm that you are running PowerShell v4.0 by running the below command:

$PSVersionTable.PSVersion

Finally, I would love to repeat that this module can be used for other updates and hotfixes. All you need is to include the appropriate KB number into your command.

Enjoy!

Reporting IP Addresses

Hi folks,

I would love to share with you helpful script which can be used to create report of the IP addresses configuration on Windows client or server. Of course ipconfig /all can be used. In this sense PowerShell script is very helpful as it talks to WMI and retrieves all the possible settings. Potentially they can be exported to CSV file by using Export-CSV cmdlet.

I found this this article to be extremely helpful as it covers multiple scenarios. In my cases I wanted to retrieve IP address and DNS related settings. Below is the code that I used them:

Get-WmiObject -Class Win32_NetworkAdapterConfiguration -Filter IPEnabled=TRUE -ComputerName . | Select-Object IPAddress,IPSubnet,DefaultIPGateway,*DNS*

I have got the following output:

Depending on the properties you are trying to retrieve, you can tweak properties as you desire.

There are couple of comments that would love to make about the output that I have got:

"Register this connection addresses in DNS" setting in GUI corresponds to the FullDNSRegistrationEnabled

"Use this connection's DNS suffix in DNS registration" corresponds to DomainDNSRegistrationEnabled

Tick box enabled corresponds to the True in output while check box disabled corresponds to False.


Enjoy!

What Subnets Are In My Site

Hi folks,

You have probably read about retrieving domain controller in the site where computer or server is located. You can read this post here. In that post I tried to

I have capitalized on this blog post from Microsoft which addresses generic situation when you try to retrieve information about AD subnets in ad generic AD site (with the default name).

In my scenario I was trying to understand which subnets are configured in the AD site where my computer (server) is located.

The script queries WMI for the value of the AD site name where computer is located and feeds it to the $SiteName variable. Then it queries domain controllers by using $SiteName as a variable in order to retrieve domain controllers located in the same AD site:


$siteName =  [System.DirectoryServices.ActiveDirectory.ActiveDirectorySite]::GetComputerSite().Name

$configNCDN = (Get-ADRootDSE).ConfigurationNamingContext

$siteContainerDN = (“CN=Sites,” + $configNCDN)

$siteDN = “CN=” + $siteName + “,” + $siteContainerDN

$siteObj = Get-ADObject -Identity $siteDN -properties “siteObjectBL”, “description”, “location”

foreach ($subnetDN in $siteObj.siteObjectBL) {

    Get-ADObject -Identity $subnetDN -properties “siteObject”, “description”, “location” |select Name,SiteObject

}

I hope you find it helpful.

Enjoy

Adding Multiple Computer Objects to AD Group

Hi folks,

Just wanted to share with you about an adventure that I had when trying to add multiple computer objects for my server into an AD group. Let us imagine that we have set up naming convention where names for computer objects are well organized.

Add-ADGroupMember is the one to be used when you're adding your server objects into AD group and you can read about it here.  A command to add a single server object to a group is quite straightfoward:

Add-ADGroupMember -Identity SvcAccPSOGroup -Members SQL01

And it seems to be straightforward when you're trying to add more than 1 server, like below:

Add-ADGroupMember -Identity SvcAccPSOGroup -Members SQL01,SQL02

However, when I tried it I got an error. Therefore I decided to run my server objects one-by-one. To avoid typing command for each server objects I queried AD for all of them and fed to the $Mem variable. Please note that I'm using well defined naming convention and therefore it's simple for me to do so. After this I'm running the Add-ADGroupMember command against every server in the $Mem variable. The final code is as below:


 $Mem = Get-ADComputer -Filter {Name -like "MYSERVER00*"}
 $Mem |foreach {Add-ADGroupMember -Identity "MY Group Name" -Members $_.DistinguishedName}

And finally you can confirm group membership by executing the following command:

 Get-ADGroupMember -Identity "MY Group Name"

Now let's consider yet another scenario. Imagine you have mailbox (or CAS) servers in your Exchange platform that share part of common name while the other part varies depending on the AD site. Below code can be run from Exchange management shell with the AD PowerShell module loaded. First we retrieve Exchange servers of a certain role (Mailbox in my example) and feed them into a variable just as above, and this is where Exchange Management shell is in play. The we will add them to the group where they should be added.

The code will be as below:

$MbxMem = Get-MailboxServer EXCH* |foreach {Get-ADComputer -Identity $
_.Name}
 $MbxMem |foreach {Add-ADGroupMember -Identity "MY Group Name" -Members $_.DistinguishedName}


I hope you'll find it helpful.

Enjoy!

Quickly Retrieving Local User Accounts

Hi folks,

Just wanted to share with you a quick way to retrieve local users (whether admin or non-admin) which exist on your computer or server. I found this article which explains how to do this. Out of the methods I used WMI method and for the parameters that I was outputting were Caption, Domain (the latter should be enough), as in my report I was trying to show the computer name where I was running the code.

So my code became as below:

Get-WmiObject -Class Win32_UserAccount -Filter  "LocalAccount='True'" | Select PSComputername, Name, Status, Disabled, AccountType, Lockout, PasswordRequired, PasswordChangeable, Caption, Domain 

I hope you find it helpful.

Enjoy!

Temporary Server Error and HOSTS file

Hi folks,

Just wanted to share with you about yet another adventure I faced. When some of the application servers that were configured to use my Exchange 2013 servers farm behind the load balancer attempted to send email via them they were prensented with the following error:

451 4.7.0 Temporary server error. Please try again later. PRX5 Downstream

As I was talking to my older brother Google when investigating this issue all the articles including this one from Dell were pointing to the known bug with the Exchange 2013. Recommendations were to update receive connectors to listen on the primary IP address of the NIC and also update HOSTS file to include IP addresses of Exchange server there.

I personally haven't got to that point because I was more lucky. When I edited HOSTS file I found there weird IP addresses (outside of my IP address range) and Exchange servers' hostnames next to them. I removed all incorrect records from the HOSTS file and rebooted the server after which my applications were able to use SMTP on Exchange to send messages.

Therefore my advise if you have this error first check your HOSTS file for strange records before you try any further troubleshooting.

Enjoy.

Add All Exchange 2016 Servers from the Site to DAG

Hi folks,

Just wanted to share with you with a script on how to quickly add new Exchange 2016 servers into DAG. Let's imagine scenario when you have introduced Exchange 2016 CU1 servers into an AD site which currently contains Exchange 2010 and 2013 servers. Let's also imagine that we have a proper and consistent naming convention (believe me, folks, it makes life sooooo easy). And of course let's assume that we have a good network infrastructure and fast AD replication.

If all of the above is truth and you are lazy to type command for each server or click Next button million of times then the below code is exactly for you. I made this script interactive so that you will need to type the name of your DAG (perhaps you have more than one AD site where you introduce Exchange servers with a different DAG in it). By this you won't need to edit this script every time when you move from site to site. Then you feed all your Exchange 2016 servers with the version "Version 15.1 (Build 396.30)" into the variable and then add them one by one. To allow AD replication occur (you know sometimes Microsoft Clustering solution are a bit temperamental and sensitive to the AD) I added 20 min for replication after each servers are added. This will allow you to kick of the script and have some time for drinking tea or reading TechNet :).

The code will look something like below:


$DAG = Read-Host "Type your DAG Name. For example: DAG01"
$ExchBox = Get-ExchangeServer $DAGSrv* | Where-Object {$_.AdminDisplayVersion -like "Version 15.1 (Build 396.30)"}
   $ExchBox |foreach {
                                      Start-Sleep -Seconds 600
                                       Add-DatabaseAvailabilityGroupServer -Identity $DAG -MailboxServer $_.Name
                                       Start-Sleep -Seconds 600}

Enjoy!

Quickly Updating Domain Based Exlusions

Hi folks,

As you well know that in Exchange 2013/16 you can now configure some new conditions about which you can read here. Among them are RecipientDomainIs and SenderDomainIs which allow you to control transport rules based on the SMTP domain of a sender and a recipient of an email. Both of these attributes are multi-value attributes and are quite easy to manipulate from EAC.


Now imagine you have transport rule which manipulates email which are perform a certiain action when email is sent to one of the domains in the list. Let's say you need the same manipulation to be done also when email is sent to people from contoso.com domain. When using PowerShell by adding and removing domains while when you type something like:
Set-TransportRule "My Transport Rule" -RecipientDomainIs contoso.com

will overwrite all the current values.

Therefore something like the code below should do a magic for you.


$RecDom = (Get-TransportRule "My Transport Rule").RecipientDomainIs
$RecDom += "contoso.com"
Set-TransportRule "My Transport Rule" -RecipientDomainIs $RecDom

All you need to do is to feed the current list of SMTP domains into a variable (let's call it $RecDom). After which we update variable with a new domain (contoso.com in our case). And finally we will update RecipientDomainIs value with the updated variable. This will preserve the current list of SMTP domains as well as add the desired domain to the list.

The similar kind of manipulation can be done with the SenderDomainIs parameter as well.

Enjoy!

Quickly Add a Signle Email Address to a Recipient

Hi folks,

Just wanted to share with you a quick way to add email addresses to Exchange recipients by using PowerShell. This is in case you are as lazy as I am and don't want to access recipient from the GUI tool like EAC.

The sample code is as follows for mailbox.

Set-Mailbox jsmith -EmailAddresses @{Add=’john.w.smith@contoso.com’}

The same can be applied for all kinds of recipients like contacts, distribution groups and so on.

Enjoy!

DAG and ReFS

Hi folks,

When you are working on planning/deploying Exchange 2016 there's now a new good practice which MS published in the Exchange 2016 Preferred Arcitecture about file system used by database volumes. Now database volumes should be formatted with ReFS.

In this article 2 important considerations are added. Firstly, we will need to disable integrity streams bit on the database volume. This should be done during volume configuration and format. Don't miss it, otherwise you will have to re-do all your storage and even mailbox databases in DAG. In Windows 2012 R2 it can be done by the followig sample command (of course disk numbers and volume can vary in your environment):

Get-Partition –Disknumber 5 –PartitionNumber 2 | Format-Volume -FileSystem REFS -NewFileSystemLabel Volume3 -SetIntegrityStreams $false

Secondly, you will need to configure your DAG to user ReFS by as the file system. We need this to ensure that spare volume when used by AutoReseed is formatted by ReFS, rather than NTFS.

By default, file system for DAG is configured to be NTFS:

To confiure your DAG with ReFS as file system you will need to run the follwing command:

Set-DatabaseAvailabilityGroup NameOfYourDAG -FileSystem ReFS

I hope it will be extremely helpful for your DAG planning and deployment.

Enjoy!

Quickly Retrieving Local Admins Group Members

Hi folks:

Just wanted to share with you the PowerShell code that can be used to retrieve members of the local admins group on your Windows server or client machine. I found it at this link and found it extremely useful.

The code to retrieve it can be

Get-WmiObject win32_groupuser |Where-Object { $_.GroupComponent -match ‘administrators’ } | ForEach-Object {[wmi]$_.PartComponent }

This provides detailed report as below (which even includes SID for each admin user):

Finally you can always adjust section { $_.GroupComponent -match ‘administrators’ } to any other group name in order to retrieve appropriate members.

Enjoy!

Add Path to Exchange Scripts to Path Variable

Hi folks,

As you know when Exchange 2010 and later is installed on your Windows server it comes with a bunch of scripts that are used for configuration and maintenance of Exchange Server. They are usually stored in C:\Program Files\Microsoft\Exchange Server\v15\Scripts (for Exchange 2013/2016) or in C:\Program Files\Microsoft\Exchange Server\v14\Scripts (for Exchange 2010). I have slightly covered this subject in this blog post.

In order to run them you will navigate to the above folder or you can access them via $exscripts variable from Exchange Management Shell.

To simplify access to scripts you can add the folder where scripts are located to the system variable named Path. As you well know it is usually done by accessing System Variables section in the advanced properties of the system.

To make this process faster it can be achieved by executing the below script. Firstly, it's worthy to mention that directories in Path variable are stored in the Path value of the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment registry key.

The script extracts list of folders in the Path value and stores it in the $oldpath variable. Then it creates new variable $newPath which consists of all directories from the $oldPath and folder where Exchange scripts are stored. After this Path value in the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment’ is updated with all the directories stored in the $newPath variable.

The code will look like below.

$oldPath=(Get-ItemProperty -Path ‘Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment’ -Name PATH).Path

$newPath=$oldPath+’;C:\Program Files\Microsoft\Exchange Server\v15\Scripts’

Set-ItemProperty -Path ‘Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment’ -Name PATH –Value $newPath

Please note that I have tested this script with elevate privileges. And I recommend you running it under the same. After script execution you will need to restart the system and it will do the magic.

After restart just start your EMS and type first couple letters of any of Exchange scripts (for example RedistributeActiveDatabases.ps1) and hit Tab button. EMS should resolved it as below:

Enjoy!

What Domain Controllers are in My Site

Hi folks,

Below is a quick script to retrieve domain controllers that are located in the same site as a computer on which your computer or server is located.

This is required for scenarios when you are trying to understand which DCs can be used for LDAP/GC queries by your computer or server.

So basically what it does it loads value of the AD site name where computer is located and feeds it to the $ADSite variable. Then it queries domain controllers by using $ADSite as a variable in order to retrieve domain controllers located in the same AD site:

$ADSite = [System.DirectoryServices.ActiveDirectory.ActiveDirectorySite]::GetComputerSite().Name
Get-ADDomainController -Filter {Site -eq "$ADSite"}

I hope you can find it helpful.

Enjoy!

Resolving Transport Service Pile Up related to DNS Issues

Hi folks,

Just wanted to share with you about my recent adventure. This applies to Exchange server versions 2010, 2013 and 2016. I have noticed pile up of the messages in the queues.There were thousands and thousands of them. Queues were in the Retry state with the following error:

The error was like the one below:

last error 4.4.0 DNS query failed. the error was:DNS query failed with error Error Retry

It was interesting as I had absolutely healthy DNS. Suspending and retrying queues didn't do its magic. So I went to the Event Viewer to find the following errors in the Application Log for Transport service (error 205 was followed by error 16025):

Investigation lead me to this TechNet blog post and this forum thread. The rest of the blog posts were similar to the former one. However the latter was the magic silver bullet to solve my problem.

I have checked my advanced IPv4 address configuration to discover the following:

After checking the "Register this connection's address in DNS" and restarting Microsoft Exchange Transport service on the affected server functionality of transport server. the above error disappeared and transport queues were returned into Ready and Active states with the fully restored mail flow.

Therefore, dear friends, my recommendation would be first checking DNS settings and should it not be the case then move to tweaking your transport service related DNS settings as advised by MS and other bloggers.

By the way, if anyone is curious what 00000000-0000-0000-0000-000000000000 as a NIC GUID means, it means that Exchange Transport Service is using all DNS adapters for DNS lookups.

And of course my special thanks to the author of the last comment in the forum thread. Sometimes problem can be too small and too hidden to spot it straight away and this makes the greatest fun.

Enjoy!

Great Script for Managing NTFS Permissions

Hi folks,

I would love to share with you about one of my recent findings, namely PowerShell module for managing NTFS file and folder permissions. Its is available on the TechNet script gallery.You can download NTFSSecurity Module from this link. Here you can read about some basics about how to manage NTFS permissions. Actually there is one more arcticle and you can find its link at the download link.

After module is downloaded (in form of zip file) you will need to deploy it on your Windows server or client box (I played with the former thought, so I leave client to you).

To install NTFSSecurity Module on your Windows 2012 server you will need to unzip files and copy all of them to %Windir%\System32\WindowsPowerShell\v1.0\Modules. If you are as lazy as I am below is the code to do it automatically (of course you may have a different folder from where you are copying your module files:

New-Item "C:\Windows\System32\WindowsPowerShell\v1.0\Modules\NTFSSecurity" -Type Directory
Copy-Item D:\\Scripts\NTFSSecurity\* "C:\Windows\System32\WindowsPowerShell\v1.0\Modules\NTFSSecurity"

After which you can
Import-Module NTFSSecurity

Below are some Sample Commands that I played with:

View NTFS permission:
Get-Item C:\Folder |Get-NTFSAccess

Add Full Access to one of the well-known Windows SIDs:
New-Item G:\Folder -Type Directory
Add-NTFSAccess -Path C:\Folder -Account 'NT AUTHORITY\NETWORK SERVICE' -AccessRights FullControl -AppliesTo ThisFolderSubfoldersAndFiles

Add full access to a domain user named john.smith:
Add-NTFSAccess -Path C:\Folder -Account 'contoso\john.smith' -AccessRights FullControl -AppliesTo ThisFolderSubfoldersAndFiles

Isn't it cool? As I love saying PowerShell is indeed Power Shell!

Enjoy

Scripts for Starting and Resuming Maitenance Mode on Exchange 2013 and 2016 servers Mailbox Servers

Hi folks,

All of us are facing tasks when we need to put our Exchange server box into maintenance mode. Usually this is done when we are installing updates or need to restart the servers (in majority of cases these two walk hand by hand). In Exchange 2010 there were 2 built-in scripts for this purpose StartDagServerMaintenance.ps1 and StopDagServerMaintenance.ps1. What each of these does can be read from their names.

However both Exchange 2013 and 2016 are different kind of animal compared to Exchange 2010 and in order to place them on maintenance you will need to disable more than just DAG and Windows cluster related settings. You will also need to drain transport queue and redirect all messages to other mailbox server (yes, mailbox servers are now responsible for transport). And when server is back up and running after maintenance you will need to revert it back from maintenance. It involves a lot of  commands and some of them can be missed or omitted. You can read more about it here.

Fortunately there is a solution for Exchange techies, especially those who are as lazy as I am. There are 2 scripts that were created for start and stop of Exchange server maintenance and they are available for download at below links:
- Start Exchange 2013/16 box maintenance
- Stop Exchange 2013/16 maintenance

I used them in my lab where I wanted to upgrade my Exchange 2016 Preview boxes to Exchange 2016 CU1.

First I placed both of them into Exchange scripts folder that is located at the installation path and renamed files to exclude version number.

Please note that if you have restricted PowerShell execution policy in your environment you may need to temporarily disable it by running

Set-PowerShellExecutionPolicy Unrestricted

After this you can run your script as follows:

cd $exscripts
.\Start-ExchangeServerMaintenanceMode.ps1 -Server Server1.domain.com -TargetServerFQDN Server2.domain.com

Script will magically put offline all the components which should be disabled and inactivated.

After it has been executed run the following command to ensure that all the components are inactive or offline:

Get-ServerComponentState -Identity Server1.domain.com

After I saw that all the critical components were inactive I have started upgrade of my Exchange 2016 Preview installation to CU1. Of course you know the command:

Setup.exe /m:Upgrade /IAcceptExchangeServerLicenseTerms

After successful upgrade I have recovered server from the maintenance mode by running the following code:

cd $exscripts
.\Stop-ExchangeServerMaintenanceMode.ps1 -Server Server1

After server has been returned back to functionality I have checked components by running Get-ServerComponentState command again (you can also run Test-Servicehealth cmdlet, however it reports of the state only of the back end components, totally ignoring frond end ones).

Finally you need to ensure that all DBs are activated by the DB activation preference. This can be achieved by running the following code wile in the scripts folder:

.\RedistributeActiveDatabases.ps1 -DagName DAG01 -BalanceDbsByActivationPreference -Confirm:$false

Enjoy