Easy Checking Message Tracking Log Across All Servers

Hi Folks,

This is a quick tip on how to extract information from message tracking logs. Exchange has nice GUI tools like Message Tracking which used ECP, EAC and Tracking Log Explorer. However we can use them against a single server (even it's remote) at a time.

The following commands allow you to extract message tracking information from all Hub Transport servers at the same time. Let's imagine the following scenario: you have Hub Transport server role co-located with CAS on the servers. And also you are trying to find all emails that were sent to a user named John Smith with email address john.smith@smtpdomain.com from a user Mary Jones with email address mary.jones@smtpdomain.com (of course nothing personal, these names are fictional here).

First I created variable called $Hubs which I populate with Hub Transport servers.

If you have separate Hub transport server roles you can use (and no Edge servers will be there simply because they are not present):

$Hubs=Get-TransportServer

If you have Edge servers in your Exchange topology then the following command is for you

$Hubs=Get-ExchangeServer | Where-Object {$_.ServerRole -like "HubTransport"}

And finally in my scenario where I have Edge servers in topology and Hub Transport servers also has CAS role running, I have populated my $Hubs variable this way (since I don't need edge servers there because I was looking only for internal emails):

$Hubs=Get-ExchangeServer | Where-Object {$_.ServerRole -like "ClientAccess, HubTransport"}

Afterwards I have run Get-MessageTrackingLog command against each Hub Transport server in $Hubs variable with specifying

$Hubs |foreach {Get-MessageTrackingLog -server $_.Name -Sender mary.jones@smtpdomain.com -Recipients john.smith@smtpdomain.com}

This command can be even better filtered by using -Start and -End  with dates and times specified (to search limited to certain dates) or any other parameter.You can check all the parameters for Get-MessageTrackingLog command here .

And finally report can be outputted as list (Format-List) to see every single detail for each SMTP transaction, though very hard to read or you can output it to CSV file using Export-CSV command. Something like:

$Hubs |foreach {Get-MessageTrackingLog -server $_.Name -Recipients john.smith@domain.com} |select EventId,Source,Sender,@{N="Recipients";E={$_.Recipients}},MessageSubject,TimeStamp |Export-Csv D:\Software\Output.csv

Enjoy!

Playing with RBAC and Roles

Hi folks,

Today let me share my experience with creating and managing role groups. RBAC is a very good security engine in Exchange 2010 and 2013 to manage administrative access by providing to administrative users only rights necessary for them to perform on the daily basis. In couple words a user is limited to a number of PowerShell commands he or she may execute or to certain sections in EMC or EAC he or she may access. I will not go into many details about this as you can read this article nicely provided in Technet.

In my case I was using standard roles created by Exchange 2010 by default with no customization which I was assigning to the newly created role groups.

Below commands are used to manage role groups and assignments.

First of all we create a role group (let's say for Level 1 engineers)

New-RoleGroup Level1

After this we assign role to a group. The following command allows to do it one-by-one

New-ManagementRoleAssignment -SecurityGroup Level1 -Role "Message Tracking"

We can check if rules were successfully assigned to this group:

Get-RoleGroup IPsoft-L1 |fl Roles

The above commands are quite simple and straightforward. Let me share one case when I needed to mimic one role group configuration between lab and production environment (which should be a normal case for any messaging environment and not only messaging).

For the sake of this article let's call this group Level3 as it will be serving admin needs of level 3 IT engineers to whom I don't want to assign Organization Management permissions.

First I have checked all roles assigned to the role group created in the lab:

Get-ManagementRoleAssignment -RoleAssignee Level3 |select Role,RoleAssigneeName

RoleAssigneeName corresponds to a user or a group to which a particular role is assigned. Of course as output of this command I will get the name of the same group assigned to it, but in the next step you will see how I used this to easily recreate this role in my production environment.

I used the same command, however instead of outputting it to screen I have exported results into a CSV file like below:

Get-ManagementRoleAssignment -RoleAssignee Level3 |select Role,RoleAssigneeName |Export-Csv D:\Software\Roles-i3.csv

I have ended up with the output like below:

As you can see I have 2 columns named after attributes I have exported Role,RoleAssigneeName . I will use them in the next command as parameters for -SecurityGroup and -Role.

As you can guess, I had imported username into pipeline and executed ManagementRoleAssignment against each of them with the parameters mentioned above.

Import-Csv .\Roles-i3.csv |foreach {New-ManagementRoleAssignment -SecurityGroup $_.RoleAssigneeName -Role $_.Role}

This simple command is used to add a users into a role group one-by-one:

Add-RoleGroupMember Level3 -Member "Farhad Mahmudov"

If we go back to my previous example of transferring role group between test lab and the production environment, I will show you some magic that I have done in my environment to import mailboxes from lab to production (provided your AD environment in the target environment is not populated as it was in my case)

First, I have extracted group members in my lab with the necessary details and exported detailes necessary for further mailboxes provisioning to CSV file:

Get-RoleGroupMember Level3 |Get-User |select Name,DisplayName,SamAccountName,UserPrincipalName | export-csv D:\Software\Level3.csv

I have transferred CSV file into my production environment and there provisioned mailboxes (please ensure that there are no trailing spaces after name or display name, otherwise you will have some grief of cleaning this out of CSV file). Please note that here I was using standard password for eevery admin which they were required to reset afterwards for security reasons.

$Password = Read-Host "Enter Users Password" -AsSecureString

Import-Csv C:\Scripts\Level3.csv |foreach {New-Mailbox -Name $_.Name -SamAccountName $_.SamAccountName -UserPrincipalName $_.UserPrincipalName -DisplayName $_.DisplayName -Password $Password}

Then I ran this code to simply check if users and mailboxes are created:

Import-Csv .\Level3.csv |foreach {Get-User $_.Name}

As you can guess mailboxes creation was successful. Finally, after that I ran Add-RoleGroupRoleMember command against each user of imported CSV file to add them to this group as members:

Import-Csv .\Level3.csv |foreach {Add-RoleGroupMember Level3 -Member $_.Name}

I hope this will be very beneficial in your daily messaging management.

Managing Distribution Group Administrators by using PowerShell

Hi Folks,

Here I would love share some simple code to tweak distribution groups administrators by using Exchange management shell.

Of course it is easier to do  by using Exchange Management Console or Exchange Admin Center (if you are using Exchange 2013), however sometimes you may need to automate this task or modify admins for more than one distribution list.

Distribution list administrators are controlled by ManagedBy property which in turn corresponds ManagedBy attribute (which can also be accessed and modified by using ADUC and have only 1 value) and to msExchCoManagedByLink (which is accessed via ADSIEDIT or Attributes editor tab of ADUC in advanced view) and can have multiple attributes.

If we modify ManagedBy propery by simply using Set-DistributionGroup with ManagedBy parameter specified, we will simply overwrite current value of this attribute which is not always what we want to achieve, in particular if we want to add more than one group admin to those currently configured.

This requires some PowerShell magic:

First we create a variable which contains the current value of the ManagedBy attribute (let's call it $Mng):

$Mng=(Get-DistributionGroup "My Dist Group").ManagedBy

Secondly we add name of our new admin to this variable:

$mng+="John Smith"


And finally, we use modified $Mng variable in configuring ManagedBy property of the distribution group:

Set-DistributionGroup "My Dist Group" -ManagedBy $Mng

I hope it will be useful to you!

Simple Code for Reporting Mailbox Size for a Single Database

Hi Folks,

This is quite simple PowerShell code that I want to share with you today.

It pipelines mailboxes from a DB (let's call it DB03) and reports their names, total items and total deleted items sizes in megabytes (as for small mailboxes reporting in gigabytes is not always accurate) along with the total items and total deleted items numbers and nicely imports all data to CSV file that can be then processed in MS Excel.

Please note that plain use of TotalItemSize and TotalDeletedItemSize outputs like this:

Therefore I had to convert information about the mailbox size and deleted item size so that it can be then processed in Excel. I had to use the following expressions to achieve this: @{N="MailboxSize MB";E={$_.TotalItemSize.Value.ToMB()}} and @{N="DeletedItem MB";E=$_.TotalDeletedItemSize.Value.ToMB()}}

All this results in the following script:

Get-Mailbox -Database DB03 |Get-MailboxStatistics |select DisplayName,@{N="MailboxSize MB";E={$_.TotalItemSize.Value.ToMB()}},@{N="DeletedItem MB";E=$_.TotalDeletedItemSize.Value.ToMB()}},ItemCount,DeletedItemCount |Export-Csv D:\Scripts\DB03-Mbx-Size.csv

As a result I'm getting a nice output as this one:

Enjoy

VSS and Mysteriously Lost Disk Space

Hi folks,

This tricky situation happened to me right after coming back to work place after long and wonderful weekend.

I have been reported by my colleagues that our customer is having issue that with the disk space. Database space was shrinking and for some databases it had reached threshold when it had to be split by 2 parts. As I started investigating this case I had the same though as they did and was ready to prep RFC for database rebuilding and mailbox moves.

However I discovered that free disk space is reported differently for mount point hosting 2 copies. One copy was reporting 48% of disk space while the other one 9%. This confused me especially after I didn't see anything special in the file structure even after hidden files were enabled. Not to mention, that LUN size for both database copies was absolutely the same, as my customer is pretty standardised.

Helpless, I went to my brother Google. And after some time I found this article which became that magic silver bullet to resolve my problem. It had quickly reminded me that we had backups failing for lengthy period of time (as you know since Exchange 2010 there's no more online backup APIs supported, only VSS). And these failed backups would not delete VSS snapshots they would have created.

This is an easy three-step solution what reveals and deletes all of the VSS copies that piled up in the storage.

First we launch the utility. As you understand it's quite obvious. You simply need to type diskshadow and hit Enter.

To type next 2 commands you will need capital letter (press CAPS LOCK or hold Shift button all the time you type them).

First we list if there are any shadow copies that were not removed by backups:

LIST SHADOWS ALL

After we discovered that there are any shadow copies sitting on your disk for a long time, which can be easily figured out by the date of each VSS snapshot, we delete all of them by running:

DELETE SHADOWS ALL

When done you will need to type exit to exit utility and then close Command Prompt.

I hope this article helpful as you solve this tricky and easy, at the same time, problem.

Enjoy.

Decomissioning Obsoltete SMTP Domains

Hi folks,

I have recently helped my client to decommission one of their obsolete SMTP domains.

For the sake of this article let's call SMTP domain ancientdomain.com.

My special thanks to the author of this article which helped me to do the trickiest bit e.g. removing aliases from bunch of mailboxes.

My client had quite a few mailboxes which were assigned aliases from which I had to scrap this out. It's worthy to mention that these mailboxes were stored in particular databases so I used -Database parameter when creating variables for the mailboxes. If you have a different scenario, you may want to use a different filter. For example: -ResultSize Unlimited (as in the original article) if you want to scrap it out of all mailboxes in your organization.

By this code I have removed aliases from mailboxes stored in the particular database:

$Mailboxes = Get-Mailbox -Database DB03
$Mailboxes | foreach{
    for ($i=0;$i -lt $_.EmailAddresses.Count; $i++)
    {
        $address = $_.EmailAddresses[$i]
        if ($address.IsPrimaryAddress -eq $false -and $address.SmtpAddress -like "*ancientdomain.com" )
        {
            Write-host($address.AddressString.ToString() | out-file d:\scripts\addressesRemoved.txt -append)
            $_.EmailAddresses.RemoveAt($i)
            $i--
        }
    }
    Set-Mailbox -Identity $_.Identity -EmailAddresses $_.EmailAddresses}


Another nice feature of this code is out-file d:\scripts\addressesRemoved.txt -append which outputs list of removed aliases thus giving us reporting and control over which domains were removed from the mailboxes.

Ok, the first and the trickiest bit is done. Now we need to prevent organization receiving emails to the ancientdomain.com. To do this we need to remove accepted domain and ensure that accepted domain removal is synced to Edge Transport servers (as in my environment, yours may differ especially if you have a different messaging hygiene solution, still you will need to remove it from there).

The first bit is straightforward and needs really no comments:

Remove-AcceptedDomain ancientdomain.com

In case of edge servers I didn't want to wait till next Edge Sync and forced it. As a lazy person I scripted it into the following code so that it runs against all Edge Transport servers in the organization:

$Edge = Get-ExchangeServer | Where-Object {$_.ServerRole -eq "Edge"}
$Edge | foreach {Start-EdgeSynchronization -Server HUBSERVER -TargetServer $_.Name}

And that's the whole magic.

Figting High CPU Usage on Mailbox Servers Caused By Uneven Spread of Mailbox Database Copies

Hi folks,

Sounds like a time to share my recent adventure.

I have recently run into an interesting issue. On 2 of servers that are the part of Database Availability Group (DAG) one was running at the top of CPU. It would never go below 70% which is not healthy. At the same time its pair hosting copies was consuming only 20-30% of CPU.

I had to resolve such injustice and make sure that CPU usage is equal and within norm on both of the pairs. First of all I needed to determine how many of copies each of the pairs has mounted and why are they mounted this way.

This is where my 2 good friends came into play e.g. PowerShell and Microsoft Excel.

First of all I ran against each server hosting the copy to make sure that this number is equal:

(Get-MailboxDatabase -Server MbxSrvHostname).count

An it actually was equal.

Now I wanted to understand how copies are mounted on both servers and why are they mounted there. I ran the following command (please don't do it yourself):

Get-MailboxDatabase -Server ServerName |select Name,Server,Servers,Activation* |export-csv D:\Software\DBCopyN4P4.csv. 

And the result was quite ugly and unreadable.

I had to tweak output for Servers and ActivationPreference parameters as follows to ensure the proper output: @{N="Servers";E={$_.Servers}},@{N="ActivationPreference";E={$_.ActivationPreference}}.

My final command for outputting databases and their copies and preferences turned to be as follows:

Get-MailboxDatabase -Server ServerName |select Name,Server,@{N="Servers";E={$_.Servers}},@{N="ActivationPreference";E={$_.ActivationPreference}} |export-csv D:\Software\DBCopyN4P4.csv

I won't be able to show the output here due to ethics, but I can guarantee that this is working.

Now the time came to redistribute mailbox database copies across both members of the pair. To make my life easier I generated CSV file that contains the list of the database copies and in particular their names which I needed for configuration command:

Get-MailboxDatabase -Server ServerName |Get-MailboxDatabaseCopyStatus |Select Name |Sort Name |Export-Csv D:\Software\DbCopy.csv

I have got output as below

I have done some processing with the file. First, I removed ugly 1st row. Secondly I have added column and added preference numbers for database copies in the following format: 1-2 2-1. This is because copies were sorted in CSV file by name and the copy on the 1st server was always listed before the one on the 2nd.

Something like

Name	Preference
DB01\SERVER01	1
DB01\SERVER02	2
DB02\SERVER01	2
DB02\SERVER02	1

Or

After that the time for magic came. I have reconfigured activation of every database copy using CSV file I had created in the previous step. I imported it into pipeline and then executed Set-MailboxDatabaseCopy command against every copy in the file. Something like this:

Import-Csv D:\Software\DbCopyPref.csv |foreach {Set-MailboxDatabaseCopy -Identity $_.Name -ActivationPreference $_.Preference}

After this you can use either Move-ActiveMailboxDatabase command or RedistributeActiveDatabases.ps1 from Exchange scripts to redistribute copies based on their preference. Because I was dealing only with 2 mailbox servers from DAG and didn't want other copies/servers to be affected  I was using Move-ActiveMailboxDatabase against every mailbox database copy. Boring, but safe.

And all of this was worthy of effort. CPU usage on both servers in pair went down to 30-50%.

Hope it will be helpful.