MVP - Most Valuable Professional
rss

Just can't get enough of IT

This blog is about mostly anything in IT. But the primary focuses are Microsoft technologies like Exchange Server, Microsoft 365, Microsoft Teams, and Cloud Security.
Last updated: 2020-09-17

 

Exchange Server 2010Exchange Server 2013Exchange Server 2016Exchange Server 2019Problem

The use of Exchange Edge Transport Servers requires the synchronization of user and configuration data from internal Exchange Servers to the Edge Transport Servers. The synchronization utilizes secure LDAP (EdgeSync) to transmit the data securely and is based on an Edge Subscription.

When you create a new Edge Subscription on your internal Exchange Servers by importing the Edge Subscription XML-file, establishing the EdgeSync-connection might fail.

You will find the following error in the application event log of the internal Exchange Server:

Log Name:      Application
Source:        MSExchange EdgeSync
Event ID:      1035
Task Category: Synchronization
Level:         Error
Keywords:      Classic
Description:
EdgeSync failed to synchronize because it only supports Cryptographic API certificates. 
The local Hub Transport server's default certificate with thumbprint XYZ isn't a 
CAPI certificate. To set a CAPI certificate as the default certificate, use the 
Enable-ExchangeCertificate cmdlet with the Services parameter using the value of SMTP.
When you encounter this error I recommend removing the Edge Subscription from the internal and the Edge Transport Server. Fixing this issue will take some time and the Edge Subscription might become invalid.

 

Reason

The private key of the current Exchange Transport default certificate of the internal Exchange servers uses a CNG private key. EdgeSync requires a CAPI1 based private key.

  • CNG = Cryptography Next Generation
  • CAPI1 = Cryptographic API (already deprecated)

This problem occurs primarily when using an Enterprise Certificate Authority using certificate templates with individual template settings. 

So far, I have not seen this issue when using public certificates issued by trusted 3rd-party Certificate Authorities.

 

How do you determine, if the type of the default transport certificate is a CNG or CAPI1 certificate?

  • Log on to the internal Exchange Server where you imported the Edge Subscription file and start a new Exchange Management Shell session
  • Query the Transport Service to identify the default certificate thumbprint
Get-TransportService | ft Name,InternalTransportCertificateThumbprint
  • Open an administrative command prompt
  • Export the certificates information from the certificate store to a text file
certutil -v -store my > cert.txt
  • Open the text file using an editor tool of your choice
  • Search for the certificate thumbprint identified in step 2
    This thumbprint is the SHA1 certificate hash
  • Scroll down to the provider section to find the following two attributes
    • ProviderType
    • KeySpec

If both attribute are of the value 0, the certificate if a CNG certificate.

The section might look like this:

Unique container name: XYZ
    Provider = Microsoft Software Key Storage Provider
    ProviderType = 0
  Flags = 20 (32)
    CRYPT_MACHINE_KEYSET -- 20 (32)
    KeySpec = 0 -- XCN_AT_NONE

 

Solution

Use OpenSSL to convert the CNG certificate to a CAPI1 certificate.

Using OpenSSL requires the download of the Windows release of OpenSSL. I recommend to not install the software on the Exchange Server but a separate Windows server or your administrative desktop system. Additionally, you need the certificate with its private key as a PFX-file.

Use the following steps to convert the CNG certificate to a CAPI1 certificate.

  • Download and install OpenSSL
  • Open the OpenSSL Command Prompt
  • Navigate to the folder containing the PFX-file
  • Convert the PFX-file to a PEM-file
    The tool will query to enter the PFX password
openssl pkcs12 -in CERT.pfx -out cert.pem -nodes
  • Convert the PEM-file to a new PFX-file
    The tool will query you to set a PFX password
openssl pkcs12 -export -in cert.pem -out NEWCERT.pfx

The new PFX-file is now a CAPI1 certificate. The new certificate has the same thumbprint. Now you must replace the current certificate used by Exchange Server with the new certificate. 

Replacing the certificate requires a downtime of each Exchange Server requiring the certificate replacement. This is due to the requirement to remove the CNG certificate first, following the import of the CAPI1 certificate. Afterward, you need to enable the required Exchange services.

  • Log on to the internal Exchange Server where you imported the Edge Subscription file and start a new Exchange Management Shell session
  • Query local Exchange Server certificates and identify the thumbprint of the default Exchange Server self-signed certificate 
    The certificate common name (CN) equals the server name
Get-ExchangeCertificate -Server SERVERNAME
  • Change the Exchange Transport default certificate to the self-signed certificate before deleting the CNG certificate
# It is mandatory to answer the query for replacing the default certificate with YES
Enable-ExchangeCertificate -Thumbprint THUMBPRINT -Services SMTP

# Restart the transport service
Restart-Service MSExchangeTransport
  • Remove the CNG certificate
    • Use either the certificate store MMC, Exchange Management Shell, or Exchange Admin Center
  • Import the CAPI1 certificate
    • Use either the certificate store MMC, Exchange Management Shell, or Exchange Admin Center
  • Enable the imported certificate and replace the default transport certificate
# It is mandatory to answer the query for replacing the default certificate with YES
Enable-ExchangeCertificate -Thumbprint NEWCERTTHUMBPRINT -Services SMTP

# Restart the transport service
Restart-Service MSExchangeTransport
  • Repeat the certificate replace for each Exchange Server in the same Active Directory site

 

Now, that you updated the local Exchange Servers there is one more step that needs to be checked on the Edge Transport Servers.

Edge Transport Servers are not domain-joined and therefore do not receive any GPO-based configuration. Each required configuration must be performed locally. To ensure that the default transport certificate of the internal Exchange servers can be used for cryptographic operations we must ensure that the certificate chain of that certificate is present in the certificate store of Edge Transport servers.

Take a look at the certificate chain of the converted CAPI1 certificate and import the Root-CA and Subordinate-CA certificates into the Edge Transport servers local certificate store. You must ensure that the certificates are placed into appropriate stores:

  • Root-CA certificate goes into Trusted Root Certification Authorities \ Certificates
  • Subordinate-CA certificate goes into Intermediate Certification Authorities \ Certificates

 

Next, you create a new Edge Subscription on your Edge Transport server and create a new subscription for the Active Directory site on the internal Exchange Server. The internal Exchange Servers are now able to establish an EdgeSync connection and encrypting the data transferred to the Edge Transport servers.

 

Note

When you receive the TLS certificate as PFX/PKCS12 file, you import the certificate and the private key. The import process itself defines the priavte key Crypto Provider. Using the following command line you ensure that the import process suses the legacy crypto provider.

certutil -csp "Microsoft RSA SChannel Cryptographic Provider" -importpfx my MYCERTpfx

 

Links

 

Enjoy Exchange Server and Edge Transport!

 

 

Read More »

This is a post summarizing the configuration values for important Exchange-related Active Directory object attributes.

Whenever you need to look up these values for troubleshooting, or editing the values manually.

Note: You should not edit any of the values manually, just because you can. Edit any Exchange-related attributes, if you are familiar with the result of your changes.

 

RemoteRecipientType

Attribute

  • msExchRemoteRecipientType 

 

1
ProvisionMailbox
2
ProvisionArchive (On-Premises Mailbox)
3
ProvisionMailbox, ProvisionArchive
4
Migrated (UserMailbox)
6
ProvisionArchive, Migrated
8
DeprovisionMailbox
10
ProvisionArchive, DeprovisionMailbox
16
DeprovisionArchive (On-Premises Mailbox)
17
ProvisionMailbox, DeprovisionArchive
20
Migrated, DeprovisionArchive
24
DeprovisionMailbox, DeprovisionArchive
33
ProvisionMailbox, RoomMailbox
35
ProvisionMailbox, ProvisionArchive, RoomMailbox
36
Migrated, RoomMailbox
38
ProvisionArchive, Migrated, RoomMailbox
49
ProvisionMailbox, DeprovisionArchive, RoomMailbox
52
Migrated, DeprovisionArchive, RoomMailbox
65
ProvisionMailbox, EquipmentMailbox
67
ProvisionMailbox, ProvisionArchive, EquipmentMailbox
68
Migrated, EquipmentMailbox
70
ProvisionArchive, Migrated, EquipmentMailbox
81
ProvisionMailbox, DeprovisionArchive, EquipmentMailbox
84
Migrated, DeprovisionArchive, EquipmentMailbox
100
Migrated, SharedMailbox
102
ProvisionArchive, Migrated, SharedMailbox
116
Migrated, DeprovisionArchive, SharedMailbox

 

Recipient Type 

Attribute

  • msExchRecipientDisplayType

 

Display Type
msExchRecipientDisplayType
(Decimal Value)
RecipientType
Mailbox User
0
MailboxUser
Distribution Group
1
DistrbutionGroup
Public Folder
2
PublicFolder
Dynamic Distribution Group
3
DynamicDistributionGroup
Organization
4
Organization
Private Distribution List
5
PrivateDistributionList
Remote Mail User
6
RemoteMailUser
Conference Room Mailbox
7
ConferenceRoomMailbox
Equipment Mailbox
8
EquipmentMailbox
ACL able Mailbox User
1073741824
ACLableMailboxUser
Security Distribution Group
1043741833
SecurityDistributionGroup
Synced Mailbox User
-2147483642
SyncedMailboxUser
Synced UDG as UDG
-2147483391
SyncedUDGasUDG
Synced UDG as Contact
-2147483386
SyncedUDGasContact
Synced Public Folder
-2147483130
SyncedPublicFolder
Synced Dynamic Distribution Group
-2147482874
SyncedDynamicDistributionGroup
Synced Remote Mail User
-2147482106
SyncedRemoteMailUser
Synced Conference Room Mailbox
-2147481850
SyncedConferenceRoomMailbox
Synced Equipment Mailbox
-2147481594
SyncedEquipmentMailbox
Synced USG as UDG
-2147481343
SyncedUSGasUDG
Synced USG as Contact
-2147481338
SyncedUSGasContact
ACL able Synced Mailbox User
-1073741818
ACLableSyncedMailboxUser
ACL able Synced Remote Mail User
-1073740282
ACLableSyncedRemoteMailUser
ACL able Synced USG as Contact
-1073739514
ACLableSyncedUSGasContact
Synced USG as USG
-1073739511
SyncedUSGasUSG

 

 

  • Exchange Server: msExchRecipientTypeDetails
  • Exchange Online: RecipientTypeDetails

 

Object Type
msExchRecipientTypeDetails
(Decimal Value)
RecipientTypeDetails
User Mailbox
1
UserMailbox
Linked Mailbox
2
LinkedMailbox
Shared Mailbox
4
SharedMailbox
Legacy Mailbox
8
LegacyMailbox
Room Mailbox
16
RoomMailbox
Equipment Mailbox
32
EquipmentMailbox
Mail Contact
64
MailContact
Mail User
128
MailUser
Mail-Enabled Universal Distribution Group
256
MailUniversalDistributionGroup
Mail-Enabled Non-Universal Distribution Group
512
MailNonUniversalGroup
Mail-Enabled Universal Security Group
1024
MailUniversalSecurityGroup
Dynamic Distribution Group
2048
DynamicDistributionGroup
Public Folder
4096
Public Folder
System Attendant Mailbox
8192
SystemAttendantMailbox
System Mailbox
16384
SystemMailbox
Cross-Forest Mail Contact
32768
MailForestContact
User
65536
User
Contact
131072
Contact
Universal Distribution Group
262144
UniversalDistributionGroup
Universal Security Group
524288
UniversalSecurityGroup
Non-Universal Group
1048576
NonUniversalGroup
Disabled User
2097152
DisabledUser
Microsoft Exchange
4194304
MicrosoftExchange
Arbitration Mailbox
8388608
ArbitrationMailbox
Mailbox Plan
16777216
MailboxPlan
Linked User
33554432
LinkedUser
Room List
268435456
RoomList
Discovery Mailbox
536870912
DiscoveryMailbox
Role Group
1073741824
RoleGroup
Remote Mailbox
2147483648
RemoteMailbox
Team Mailbox
137438953472
TeamMailbox

 

 

Read More »

Logo - Microsoft 365 Virtual ConferenceThe Microsoft 365 Virtual Marathon is happening on May 27-28 2020.

This is a free online event, providing you with 36 hours of non-stop sessions from speakers around the globe. You can join every time. 

  • 300+ Speakers
  • 400+ Sessions
  • Keynotes by Jeff Teper, Naomi Moneypenny, Bill Baer, Jon Levesque, Laurie Pottmeyer, and Michael Holste
  • Sessions are primarily in Englisch, but there are session in six additional languages
    • French, German, Japanese, Korean, Portuguese, and Spanish 
  • Hashtag: #M365VM

The virtual conference is a joint effort with SPC.

 

I speak at Microsoft 365 Virtual Marathon about:

  • Exchange Hybrid - What, Why, and How
  • Wednesday, May 27th
  • 23:00h (CEST)

 

The marathon session plan is available here.

Register now.

 

Link

 

Enjoy!

 

 

Read More »
Last updated: 2020-03-28
First posted: 2020-04-01


Exchange Server 2016Exchange Server 2019Once upon a time at an Exchange Conference near you, a member of the Exchange Product Group (PG) announced that the very last Exchange Server will go away when having an active Exchange hybrid setup.

This was a hot topic for discussions at the Microsoft Exchange Conferences (MEC, @IamMEC) in 2012 and 2014, already. Since then the Exchange PG came up with a number of reasons why this is not possible. The question on when we will finally be able to remove the very last Exchange Server from the on-premises Exchange organization was asked every year at the Ignite Conference. 

 

Current Situation

Currently, the supported scenario for hybrid configurations between your on-premises Exchange organization and Exchange Online requires that you keep the last Exchange Server for creating, and managing Exchange related objects, even if those objects are located in Exchange Online. 

The following diagram illustrates the current requirements:

  • Domain Controller Servers, with the most recent Active Directory Schema Update for a supported Exchange Server version, and stored Exchange-extended AD objects.
  • DirSync Server, with AAD Connect, synchronizing Exchange-extended AD objects stored in on-premises Active Directory with Azure AD.
  • Last Exchange Server, running a supported version of Exchange Server, managing the objects stored in on-premises Active Directory.

 

Exchange Classic Hybrid Configuration Model

 

 

NOTE
The last Exchange server requires that you keep it updated in regard to monthly Windows Server updates, and quarterly Exchange Server Cumulative Updates. Keep in mind that the Exchange Server follows a strict N-2 approach, to remain in a supported operational state. This means that only the most current Cumulative Update (CU) for modern Exchange versions, or Update Rollup (UR) for the good ole legacy version Exchange Server 2010, keep your on-premises Exchange organization on a supported state.

 

Interim solutions

In the past, there was communication on certain interim solutions that were supposed to support you in removing the last Exchange Server from your Exchange organization. Such interim solutions were:

  • ADSI Edit
  • Identity Management Solutions (IDM)
  • PowerShell

At Ignite those solutions even made it into the official session catalog:

 

All those interim solutions leave your on-premises Exchange organization and the Active Directory configuration in an uncomfortable twilight-zone. It was still something that worked somehow, but you knew it was officially not supported, and the secure and stable operation of the hybrid configuration was at risk.

 

But wait...

Removing the last Exchange Server is supported!

(at least when all components are released)

 

 

Solution

Exchange Server 2019The new approach for managing your Exchange Online tenancy after migrating your on-premises Exchange organization to Exchange Online does not require an on-premises Exchange Server. 

The new mode of operation reduces your on-premises requirements to:

  • Domain Controller Servers, running the most recent release Windows Server 2019 with the most recent Active Directory Schema Update for Exchange Server 2019 CU 5. This updates the Exchange-extended AD objects and the hybrid configuration object to support modern EXO management. Also, you must enable the Windows Feature "Exchange Online Remote Features" as described below.
     
  • DirSync Server, with AAD Connect, synchronizing Exchange-extended AD objects stored in on-premises Active Directory with Azure AD. You must run the most recent release of Azure AD Connect (1.4.38.0) and have Exchange Hybrid enabled in AAD Connect.
    See release notes for Azure AD Connect
     
  • Administrative PC, running the most recent version of the Exchange Online PowerShell v2 (aka ExchangeOnlineManagement)

 

The following diagram illustrates the new modern Exchange Online Management experience:

 

Modern Exchange Management Experience

 

Simply you remove the requirement to use on-premises Exchange Server to write to your on-premises Active Directory. Instead, Azure AD Connect uses a new synchronization capability to handle the new Exchange Management experience in the AAD Connect MetaVerse. The on-premises AD-connector writes the changes to Active Directory which keeps the Active Directory up-to-date for all other on-premises solutions that require identities to have a proper state.

You execute all Exchange-related actions using the new Exchange Online Management PowerShell module, or, if needed, the new Modern Exchange Admin Center (EAC, which was announced at Ignite 2019.

 

Enable Modern Exchange Admin Experience (MEAE)

WARNING
The following actions might interfere with the current configuration of your Exchange organization and Active Directory forest. Therefore, I highly recommend that you put yourself in a comfortable position and apply the changes described below in a test/lab environment before you deploy the changes into the production environment.


Before you uninstall the last Exchange Server from your on-premises Exchange organization, ensure that you

  • Updated the Active Directory Schema Version to Exchange Server 2019 CU5
  • Prepare the Active Directory and all domains in the Active Directory Forest that contain Exchange objects
  • Ensure that there are not pre-Windows 2019 domain controller in use, and patch all domain controller to the March 2020 release
  • Install the new EXORemote Windows Feature 
PS C:\> Get-WindowsFeature

Display Name                                            Name                       Install State
------------                                            ----                       -------------
[ ] Active Directory Certificate Services               AD-Certificate                 Available
    [ ] Certification Authority                         ADCS-Cert-Authority            Available
    [ ] Certificate Enrollment Policy Web Service       ADCS-Enroll-Web-Pol            Available
    [ ] Certificate Enrollment Web Service              ADCS-Enroll-Web-Svc            Available
    [ ] Certification Authority Web Enrollment          ADCS-Web-Enrollment            Available
    [ ] Network Device Enrollment Service               ADCS-Device-Enrollment         Available
    [ ] Online Responder                                ADCS-Online-Cert               Available
[ ] Active Directory Domain Services                    AD-Domain-Services             Available
[ ] Active Directory Federation Services                ADFS-Federation                Available
[ ] Active Directory Lightweight Directory Services     ADLDS                          Available
[ ] Active Directory Rights Management Services         ADRMS                          Available
    [ ] Active Directory Rights Management Server       ADRMS-Server                   Available
    [ ] Identity Federation Support                     ADRMS-Identity                 Available
[ ] Device Health Attestation                           DeviceHealthAttestat...        Available
[ ] DHCP Server                                         DHCP                           Available
[ ] DNS Server                                          DNS                            Available
[ ] Exchange Online Remote Features                     EXORemote                      Available
[ ] Fax Server                                          Fax                            Available
[X] File and Storage Services                           FileAndStorage-Services        Installed
    [X] File and iSCSI Services                         File-Services                  Installed
        [X] File Server                                 FS-FileServer                  Installed
        [ ] BranchCache for Network Files               FS-BranchCache                 Available
[...]

 

PS C:\> Install-WindowsFeature -Name EXORemote

Display Name                                            Name                       Install State
------------                                            ----                       -------------
[ ] Active Directory Certificate Services               AD-Certificate                 Available
    [ ] Certification Authority                         ADCS-Cert-Authority            Available
    [ ] Certificate Enrollment Policy Web Service       ADCS-Enroll-Web-Pol            Available
    [ ] Certificate Enrollment Web Service              ADCS-Enroll-Web-Svc            Available
    [ ] Certification Authority Web Enrollment          ADCS-Web-Enrollment            Available
    [ ] Network Device Enrollment Service               ADCS-Device-Enrollment         Available
    [ ] Online Responder                                ADCS-Online-Cert               Available
[ ] Active Directory Domain Services                    AD-Domain-Services             Available
[ ] Active Directory Federation Services                ADFS-Federation                Available
[ ] Active Directory Lightweight Directory Services     ADLDS                          Available
[ ] Active Directory Rights Management Services         ADRMS                          Available
    [ ] Active Directory Rights Management Server       ADRMS-Server                   Available
    [ ] Identity Federation Support                     ADRMS-Identity                 Available
[ ] Device Health Attestation                           DeviceHealthAttestat...        Available
[ ] DHCP Server                                         DHCP                           Available
[ ] DNS Server                                          DNS                            Available
[X] Exchange Online Remote Features                     EXORemote                      Installed
[ ] Fax Server                                          Fax                            Available
[X] File and Storage Services                           FileAndStorage-Services        Installed
    [X] File and iSCSI Services                         File-Services                  Installed
        [X] File Server                                 FS-FileServer                  Installed
        [ ] BranchCache for Network Files               FS-BranchCache                 Available
[...]

 

Even though not explicitly stated, you should restart the server after installing the Windows feature.

 

Modern EXO Exchange Admin Center - New Management Server

 

As part of the next AAD Connect synchronization cycle, the magic happens.

  • The added EXO modern management server is validated
    • You'll see an error sign in Modern Exchange Admin Center if the validation has failed
  •  Azure AD removes the "on-premises"-lock from the Exchange related attributes, so you can start managing these attributes in EXO
  • The Exchange hybrid configuration object is set to "Modern Coexistence", ensuring that a newly added on-premises Exchange Server will not interfere with the setup

 

Verify that you can edit the Exchange related attributes of synchronized Active Directory objects in Exchange Online or Azure AD before you remove your last Exchange Server. 

Whey ready to uninstall the last Exchange Server you must use the following command line parameters to remove the server as intended. Otherwise, you'll leave the Exchange organization in an inchoate state. Ensure that you use an administrative PowerShell session. 

./Setup.exe /mode:uninstall /SwitchToMEMA /IAcceptExchangeOnlineLicenseTerms

Normally, you do not have to accept license terms when uninstalling Exchange Server, but in this case, you have to accept the Exchange Online license terms.

 

Prerequisites

  • Windows Server 2019 Desktop Experience Editon or Core Edition, March 2020 Update
    Core Edition recommended
  • Azure AD Connect, release 1.4.38.0
  • Exchange Server 2019 CU5, as the last Exchange Server

 

Links

 

Enjoy the modern experience and management options of Exchange Online!

 


Exchange Conferences

Read More »

Exchange Server 2016Exchange Server 2019

Exchange Server 2016 introduced the PowerShell cmdlet Get-MailboxServerRedundancy. This cmdlet helps you plan and prepare for Exchange Server maintenance by querying the current maintenance readiness of the database availability group (DAG). 

Interestingly, there is no PowerShell help available for this vital cmdlet. Microsoft Docs or Exchange Management Shell's Get-Help provide any valuable information.

When querying a DAG about the server redundancy status, the cmdlet's default output provides you with the essential information.

The default output contains information about:

  • Identity
    Name of the DAG member server
     
  • IsServerFoundInAD
    Indicates if the corresponding server computer object exists Active Directory
     
  • IsInMaintenance
    Indicates if the server is currently in maintenance mode
     
  • RepairUrgency
    Indicates an aggregated state of the mailbox database and search index repair modes 
     
  • SafeForMaintenance
    Indicates if you can safely activate the maintenance mode for this server
     
  • HealthInfoLastUpdateTime
    Timestamp when the server's health state was last updated
     

 

Example - Prior Maintenance

This example shows the Get-MailboxServerRedundancy output of a six server DAG, before activating maintenance mode for server LOCEXS06.

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 09:10:11
LOCEXS02        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS03        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS06        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS05        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS04        True          False       Prohibited    False              17.02.2020 09:10:11

 

As Exchange Administrator, you are most interested in the information displayed in columns RepairUrgency and SafeForMaintenance.

Screenshot Get-MailboxServerRedundancy

 

As you can see in this screenshot, no server is in maintenance mode. Servers S01 and S04 have a RepairUrgency state of Prohibited, and a SafeForMaintenance state of False. The latter tells us that we cannot activate maintenance mode for servers safely without risking mailbox database redundancy. 

What is the reason for this? Let's have a look.

 

Server Information

You can use the same cmdlet to query detailed information for each member server of the DAG. The default output for a single server does not provide any additional information on the server status. 

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01 -Identity LOCEXS01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 09:11:11

 

Because we cannot activate maintenance mode for server LOCEXS01 safely, we are interested in identifying which redundancy state is responsible.

You can find this information by displaying the detailed server information.

 

Detailed Server Information

Use the Format-List, or short FL, cmdlet to display the Get-MailboxServerRedundancy cmdlet output as a formatted list.

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01 -Identity LOCEXS01 | FL

RunspaceId                                  : 70d82f8d-e6ca-4bfc-863f-11300a9784ff
Identity                                    : LOCEXS01
IsServerFoundInAD                           : True
IsInMaintenance                             : False
RepairUrgency                               : Prohibited
SafeForMaintenance                          : False
ServerContactedFqdn                         : LOCEXS04.VARUNAGROUP.DE
HealthInfoCreateTime                        : 15.06.2018 15:16:19
HealthInfoLastUpdateTime                    : 17.02.2020 09:11:11
ServerFoundInAD                             : CurrentState: Active; LastActiveTransition: 15.06.2018 15:22:16;
                                              LastInactiveTransition:
InMaintenance                               : CurrentState: Inactive; LastActiveTransition: 17.01.2020 09:07:02;
                                              LastInactiveTransition: 17.01.2020 10:42:02
AutoActivationPolicyBlocked                 : CurrentState: Inactive; LastActiveTransition: 09.01.2020 10:14:50;
                                              LastInactiveTransition: 09.01.2020 11:00:51
ActivationDisabledAndMoveNow                : CurrentState: Inactive; LastActiveTransition: ; LastInactiveTransition:
                                              15.06.2018 15:22:16
HighAvailabilityComponentStateOffline       : CurrentState: Inactive; LastActiveTransition: 17.01.2020 09:07:02;
                                              LastInactiveTransition: 17.01.2020 10:42:02
CriticalForMaintainingAvailability          : CurrentState: Inactive; LastActiveTransition: 31.01.2020 16:52:49;
                                              LastInactiveTransition: 31.01.2020 16:56:49
CriticalForMaintainingRedundancy            : CurrentState: Active; LastActiveTransition: 29.01.2020 11:43:06;
                                              LastInactiveTransition: 29.01.2020 11:42:06
PotentiallyCriticalForMaintainingRedundancy : CurrentState: Active; LastActiveTransition: 01.02.2020 05:49:37;
                                              LastInactiveTransition:
CriticalForRestoringAvailability            : CurrentState: Inactive; LastActiveTransition: 06.05.2019 09:16:36;
                                              LastInactiveTransition: 06.05.2019 09:20:36
CriticalForRestoringRedundancy              : CurrentState: Inactive; LastActiveTransition: 29.01.2020 11:42:06;
                                              LastInactiveTransition: 29.01.2020 11:43:06
HighForRestoringAvailability                : CurrentState: Inactive; LastActiveTransition: 29.01.2020 11:42:06;
                                              LastInactiveTransition: 29.01.2020 11:43:06
HighForRestoringRedundancy                  : CurrentState: Inactive; LastActiveTransition: 10.02.2020 09:05:02;
                                              LastInactiveTransition: 10.02.2020 09:06:02
IsSafeForMaintenance                        : CurrentState: Inactive; LastActiveTransition: 03.11.2019 09:42:35;
                                              LastInactiveTransition: 12.11.2019 06:29:58
IsValid                                     : True
ObjectState                                 : Unchanged

 

The lines 24-27 show the information we want to know. Both, the CriticalForMaintainingRedundancy and PotentiallyCriticalForMaintainingRedundancy parameters have a CurrentState value of Active. The Primary Activation Manager (PAM) considers the server availability critical to provide redundant availability of the database copies hosted by this server. 

Each of state-parameter shows three pieces of information:

  • CurrentState
    The current state, either Active  or Inactive
     
  • LastActiveTransition
    The timestamp of the last state change to Active
     
  • LastInactiveTransition
    The timestamp of the last state change to Inactive

 

I cover the different state-parameters in a future blog post. 


But there is still the bothering question of why are two of the six servers not safe for activating maintenance?

The reason is simple. The mailbox databases mounted by the member servers of the DAG have a different number of database copies. This configuration is due to data storage capacity constraints.

The mailbox databases storing primary user mailboxes use four database copies per database. Those copies are evenly distributed across all six mailbox servers. Mailbox database storing online archive mailboxes use three copies per database. This database copy layout allows for safely activating server maintenance for one server at a time without risk to database redundancy.

The servers LOCEXS01 and LOCEXS04 hold mailbox databases with just two copies per configured database. Placing one of those two servers into maintenance mode reduces the database availability for these mailbox databases to one. Therefore, PAM informs us that database redundancy is at risk when activating maintenance for those two servers. 

 

Example - During Maintenance

This example shows the member server redundancy state while LOCEXS06 is in maintenance. The reason for monthly maintenance for installing Windows updates.

Maintenance was activated using the StartDagServerMaintenance.ps1 PowerShell script.

 

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup indag01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS02        True          False       Normal        True               17.02.2020 11:04:12
LOCEXS03        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS06        True          True        High          True               17.02.2020 11:04:12
LOCEXS05        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS04        True          False       Prohibited    False              17.02.2020 11:04:12

Having a single server in maintenance has a significant impact on all other servers in the DAG. The servers LOCEXS03 and LOCEXS05 are not safe for maintenance as well. Activating maintenance for those two servers would affect the database redundancy for the databases hosted by those two servers.

 

Example - After Maintenance

After completing all maintenance tasks, e.g., installing Windows Updates or a new Exchange Server Cumulative Update, you end server maintenance using the PowerShell script StopDagServerMaintenance.ps1.

We query the server redundancy state again. 

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup indag01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 11:23:12
LOCEXS02        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS03        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS06        True          False       High          True               17.02.2020 11:23:12
LOCEXS05        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS04        True          False       Prohibited    False              17.02.2020 11:23:12

 

Server LOCEXS06 is not in maintenance, but the RepairUrgency state is High. The local Exchange Server replication engine is still busy replicating and processing log files, and updating the search indices. When CopyQueueLength and ReplayQueueLength are back to 0, and ContentIndexStates are back to Healthy, the RepairUrgency switches to Normal.

 

Tip

  • You receive an error message when activating maintenance for an Exchange Server not safe for maintenance using
    StartDagServerMaintenance.ps1 -serverName [SERVER]

    In this case, you must use:

.\StartDagServerMaintenance.ps1 -serverName SERVERNAME -overrideMinimumTwoCopies:$true

 

Enjoy Exchange Server!

 

 

Read More »