Just can't get enough of IT

The Microsoft 365 Virtual Marathon is happening on May 27-28 2020.

This is a free online event, providing you with 36 hours of non-stop sessions from speakers around the globe. You can join every time. 

• 300+ Speakers
• 400+ Sessions
• Keynotes by Jeff Teper, Naomi Moneypenny, Bill Baer, Jon Levesque, Laurie Pottmeyer, and Michael Holste
• Sessions are primarily in Englisch, but there are session in six additional languages
  • French, German, Japanese, Korean, Portuguese, and Spanish 
• Hashtag: #M365VM

The virtual conference is a joint effort with SPC.

I speak at Microsoft 365 Virtual Marathon about:

• Exchange Hybrid - What, Why, and How
• Wednesday, May 27th
• 23:00h (CEST)

The marathon session plan is available here.

Register now.

Link

• Register for Microsoft 365 Virtual Marathon

Enjoy!

When you plan to implement an Exchange Hybrid Configuration between your on-premises Exchange Organization and Exchange online you have to choose between two variants and five operating modes. It is not as complicated as it sounds.

I have written a blog post about the different options available. 

The post is published in ENow's ESE blog.

• What are your Exchange Hybrid Options?

Enjoy.

Once upon a time at an Exchange Conference near you, a member of the Exchange Product Group (PG) announced that the very last Exchange Server will go away when having an active Exchange hybrid setup.

This was a hot topic for discussions at the Microsoft Exchange Conferences (MEC, @IamMEC) in 2012 and 2014, already. Since then the Exchange PG came up with a number of reasons why this is not possible. The question on when we will finally be able to remove the very last Exchange Server from the on-premises Exchange organization was asked every year at the Ignite Conference. 

Current Situation

Currently, the supported scenario for hybrid configurations between your on-premises Exchange organization and Exchange Online requires that you keep the last Exchange Server for creating, and managing Exchange related objects, even if those objects are located in Exchange Online. 

The following diagram illustrates the current requirements:

• Domain Controller Servers, with the most recent Active Directory Schema Update for a supported Exchange Server version, and stored Exchange-extended AD objects.
• DirSync Server, with AAD Connect, synchronizing Exchange-extended AD objects stored in on-premises Active Directory with Azure AD.
• Last Exchange Server, running a supported version of Exchange Server, managing the objects stored in on-premises Active Directory.

Interim solutions

In the past, there was communication on certain interim solutions that were supposed to support you in removing the last Exchange Server from your Exchange organization. Such interim solutions were:

• ADSI Edit
• Identity Management Solutions (IDM)
• PowerShell

At Ignite those solutions even made it into the official session catalog:

• Ignite 2018 - THR2145 - Why do we need to keep an Exchange Server on-premises when we move to the cloud?

All those interim solutions leave your on-premises Exchange organization and the Active Directory configuration in an uncomfortable twilight-zone. It was still something that worked somehow, but you knew it was officially not supported, and the secure and stable operation of the hybrid configuration was at risk.

But wait...

Removing the last Exchange Server is supported!

(at least when all components are released)

Solution

The new approach for managing your Exchange Online tenancy after migrating your on-premises Exchange organization to Exchange Online does not require an on-premises Exchange Server. 

The new mode of operation reduces your on-premises requirements to:

• Domain Controller Servers, running the most recent release Windows Server 2019 with the most recent Active Directory Schema Update for Exchange Server 2019 CU 5. This updates the Exchange-extended AD objects and the hybrid configuration object to support modern EXO management. Also, you must enable the Windows Feature "Exchange Online Remote Features" as described below.
   
• DirSync Server, with AAD Connect, synchronizing Exchange-extended AD objects stored in on-premises Active Directory with Azure AD. You must run the most recent release of Azure AD Connect (1.4.38.0) and have Exchange Hybrid enabled in AAD Connect.
  See release notes for Azure AD Connect. 
   
• Administrative PC, running the most recent version of the Exchange Online PowerShell v2 (aka ExchangeOnlineManagement)

The following diagram illustrates the new modern Exchange Online Management experience:

Simply you remove the requirement to use on-premises Exchange Server to write to your on-premises Active Directory. Instead, Azure AD Connect uses a new synchronization capability to handle the new Exchange Management experience in the AAD Connect MetaVerse. The on-premises AD-connector writes the changes to Active Directory which keeps the Active Directory up-to-date for all other on-premises solutions that require identities to have a proper state.

You execute all Exchange-related actions using the new Exchange Online Management PowerShell module, or, if needed, the new Modern Exchange Admin Center (EAC, which was announced at Ignite 2019.

Enable Modern Exchange Admin Experience (MEAE)

Before you uninstall the last Exchange Server from your on-premises Exchange organization, ensure that you

• Updated the Active Directory Schema Version to Exchange Server 2019 CU5
• Prepare the Active Directory and all domains in the Active Directory Forest that contain Exchange objects
• Ensure that there are not pre-Windows 2019 domain controller in use, and patch all domain controller to the March 2020 release
• Install the new EXORemote Windows Feature 

PS C:\> Get-WindowsFeature

Display Name                                            Name                       Install State
------------                                            ----                       -------------
[ ] Active Directory Certificate Services               AD-Certificate                 Available
    [ ] Certification Authority                         ADCS-Cert-Authority            Available
    [ ] Certificate Enrollment Policy Web Service       ADCS-Enroll-Web-Pol            Available
    [ ] Certificate Enrollment Web Service              ADCS-Enroll-Web-Svc            Available
    [ ] Certification Authority Web Enrollment          ADCS-Web-Enrollment            Available
    [ ] Network Device Enrollment Service               ADCS-Device-Enrollment         Available
    [ ] Online Responder                                ADCS-Online-Cert               Available
[ ] Active Directory Domain Services                    AD-Domain-Services             Available
[ ] Active Directory Federation Services                ADFS-Federation                Available
[ ] Active Directory Lightweight Directory Services     ADLDS                          Available
[ ] Active Directory Rights Management Services         ADRMS                          Available
    [ ] Active Directory Rights Management Server       ADRMS-Server                   Available
    [ ] Identity Federation Support                     ADRMS-Identity                 Available
[ ] Device Health Attestation                           DeviceHealthAttestat...        Available
[ ] DHCP Server                                         DHCP                           Available
[ ] DNS Server                                          DNS                            Available
[ ] Exchange Online Remote Features                     EXORemote                      Available
[ ] Fax Server                                          Fax                            Available
[X] File and Storage Services                           FileAndStorage-Services        Installed
    [X] File and iSCSI Services                         File-Services                  Installed
        [X] File Server                                 FS-FileServer                  Installed
        [ ] BranchCache for Network Files               FS-BranchCache                 Available
[...]

PS C:\> Install-WindowsFeature -Name EXORemote

Display Name                                            Name                       Install State
------------                                            ----                       -------------
[ ] Active Directory Certificate Services               AD-Certificate                 Available
    [ ] Certification Authority                         ADCS-Cert-Authority            Available
    [ ] Certificate Enrollment Policy Web Service       ADCS-Enroll-Web-Pol            Available
    [ ] Certificate Enrollment Web Service              ADCS-Enroll-Web-Svc            Available
    [ ] Certification Authority Web Enrollment          ADCS-Web-Enrollment            Available
    [ ] Network Device Enrollment Service               ADCS-Device-Enrollment         Available
    [ ] Online Responder                                ADCS-Online-Cert               Available
[ ] Active Directory Domain Services                    AD-Domain-Services             Available
[ ] Active Directory Federation Services                ADFS-Federation                Available
[ ] Active Directory Lightweight Directory Services     ADLDS                          Available
[ ] Active Directory Rights Management Services         ADRMS                          Available
    [ ] Active Directory Rights Management Server       ADRMS-Server                   Available
    [ ] Identity Federation Support                     ADRMS-Identity                 Available
[ ] Device Health Attestation                           DeviceHealthAttestat...        Available
[ ] DHCP Server                                         DHCP                           Available
[ ] DNS Server                                          DNS                            Available
[X] Exchange Online Remote Features                     EXORemote                      Installed
[ ] Fax Server                                          Fax                            Available
[X] File and Storage Services                           FileAndStorage-Services        Installed
    [X] File and iSCSI Services                         File-Services                  Installed
        [X] File Server                                 FS-FileServer                  Installed
        [ ] BranchCache for Network Files               FS-BranchCache                 Available
[...]

Even though not explicitly stated, you should restart the server after installing the Windows feature.

• Connect to the modern Exchange Online Admin Center as a Global Administrator or Exchange Services Administrator and add at least one of your on-premises domain controllers to the list of modern management servers.
  https://admin.exchange.microsoft.com/#/modernmanagement

As part of the next AAD Connect synchronization cycle, the magic happens.

• The added EXO modern management server is validated
  • You'll see an error sign in Modern Exchange Admin Center if the validation has failed
•  Azure AD removes the "on-premises"-lock from the Exchange related attributes, so you can start managing these attributes in EXO
• The Exchange hybrid configuration object is set to "Modern Coexistence", ensuring that a newly added on-premises Exchange Server will not interfere with the setup

Verify that you can edit the Exchange related attributes of synchronized Active Directory objects in Exchange Online or Azure AD before you remove your last Exchange Server. 

Whey ready to uninstall the last Exchange Server you must use the following command line parameters to remove the server as intended. Otherwise, you'll leave the Exchange organization in an inchoate state. Ensure that you use an administrative PowerShell session. 

./Setup.exe /mode:uninstall /SwitchToMEMA /IAcceptExchangeOnlineLicenseTerms

Normally, you do not have to accept license terms when uninstalling Exchange Server, but in this case, you have to accept the Exchange Online license terms.

Prerequisites

• Windows Server 2019 Desktop Experience Editon or Core Edition, March 2020 Update
  Core Edition recommended
• Azure AD Connect, release 1.4.38.0
• Exchange Server 2019 CU5, as the last Exchange Server

Links

• Exchange Online Management PowerShell Module v2
• Hybrid deployment prerequisites
• Exchange Server hybrid deployments
• Install-WindowsFeature
• You must follow the instructions given in the follow article:
  Final instructions for enabling this awesome new MEMA-experience

Enjoy the modern experience and management options of Exchange Online!

Exchange Conferences

• MEC 2014 - Austin
• MEC 2012 - Orlando

Exchange Server 2016 introduced the PowerShell cmdlet Get-MailboxServerRedundancy. This cmdlet helps you plan and prepare for Exchange Server maintenance by querying the current maintenance readiness of the database availability group (DAG). 

Interestingly, there is no PowerShell help available for this vital cmdlet. Microsoft Docs or Exchange Management Shell's Get-Help provide any valuable information.

When querying a DAG about the server redundancy status, the cmdlet's default output provides you with the essential information.

The default output contains information about:

• Identity
  Name of the DAG member server
   
• IsServerFoundInAD
  Indicates if the corresponding server computer object exists Active Directory
   
• IsInMaintenance
  Indicates if the server is currently in maintenance mode
   
• RepairUrgency
  Indicates an aggregated state of the mailbox database and search index repair modes 
   
• SafeForMaintenance
  Indicates if you can safely activate the maintenance mode for this server
   
• HealthInfoLastUpdateTime
  Timestamp when the server's health state was last updated
   

Example - Prior Maintenance

This example shows the Get-MailboxServerRedundancy output of a six server DAG, before activating maintenance mode for server LOCEXS06.

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 09:10:11
LOCEXS02        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS03        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS06        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS05        True          False       Normal        True               17.02.2020 09:10:11
LOCEXS04        True          False       Prohibited    False              17.02.2020 09:10:11

As Exchange Administrator, you are most interested in the information displayed in columns RepairUrgency and SafeForMaintenance.

As you can see in this screenshot, no server is in maintenance mode. Servers S01 and S04 have a RepairUrgency state of Prohibited, and a SafeForMaintenance state of False. The latter tells us that we cannot activate maintenance mode for servers safely without risking mailbox database redundancy. 

What is the reason for this? Let's have a look.

Server Information

You can use the same cmdlet to query detailed information for each member server of the DAG. The default output for a single server does not provide any additional information on the server status. 

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01 -Identity LOCEXS01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 09:11:11

Because we cannot activate maintenance mode for server LOCEXS01 safely, we are interested in identifying which redundancy state is responsible.

You can find this information by displaying the detailed server information.

Detailed Server Information

Use the Format-List, or short FL, cmdlet to display the Get-MailboxServerRedundancy cmdlet output as a formatted list.

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup EXDAG01 -Identity LOCEXS01 | FL

RunspaceId                                  : 70d82f8d-e6ca-4bfc-863f-11300a9784ff
Identity                                    : LOCEXS01
IsServerFoundInAD                           : True
IsInMaintenance                             : False
RepairUrgency                               : Prohibited
SafeForMaintenance                          : False
ServerContactedFqdn                         : LOCEXS04.VARUNAGROUP.DE
HealthInfoCreateTime                        : 15.06.2018 15:16:19
HealthInfoLastUpdateTime                    : 17.02.2020 09:11:11
ServerFoundInAD                             : CurrentState: Active; LastActiveTransition: 15.06.2018 15:22:16;
                                              LastInactiveTransition:
InMaintenance                               : CurrentState: Inactive; LastActiveTransition: 17.01.2020 09:07:02;
                                              LastInactiveTransition: 17.01.2020 10:42:02
AutoActivationPolicyBlocked                 : CurrentState: Inactive; LastActiveTransition: 09.01.2020 10:14:50;
                                              LastInactiveTransition: 09.01.2020 11:00:51
ActivationDisabledAndMoveNow                : CurrentState: Inactive; LastActiveTransition: ; LastInactiveTransition:
                                              15.06.2018 15:22:16
HighAvailabilityComponentStateOffline       : CurrentState: Inactive; LastActiveTransition: 17.01.2020 09:07:02;
                                              LastInactiveTransition: 17.01.2020 10:42:02
CriticalForMaintainingAvailability          : CurrentState: Inactive; LastActiveTransition: 31.01.2020 16:52:49;
                                              LastInactiveTransition: 31.01.2020 16:56:49
CriticalForMaintainingRedundancy            : CurrentState: Active; LastActiveTransition: 29.01.2020 11:43:06;
                                              LastInactiveTransition: 29.01.2020 11:42:06
PotentiallyCriticalForMaintainingRedundancy : CurrentState: Active; LastActiveTransition: 01.02.2020 05:49:37;
                                              LastInactiveTransition:
CriticalForRestoringAvailability            : CurrentState: Inactive; LastActiveTransition: 06.05.2019 09:16:36;
                                              LastInactiveTransition: 06.05.2019 09:20:36
CriticalForRestoringRedundancy              : CurrentState: Inactive; LastActiveTransition: 29.01.2020 11:42:06;
                                              LastInactiveTransition: 29.01.2020 11:43:06
HighForRestoringAvailability                : CurrentState: Inactive; LastActiveTransition: 29.01.2020 11:42:06;
                                              LastInactiveTransition: 29.01.2020 11:43:06
HighForRestoringRedundancy                  : CurrentState: Inactive; LastActiveTransition: 10.02.2020 09:05:02;
                                              LastInactiveTransition: 10.02.2020 09:06:02
IsSafeForMaintenance                        : CurrentState: Inactive; LastActiveTransition: 03.11.2019 09:42:35;
                                              LastInactiveTransition: 12.11.2019 06:29:58
IsValid                                     : True
ObjectState                                 : Unchanged

The lines 24-27 show the information we want to know. Both, the CriticalForMaintainingRedundancy and PotentiallyCriticalForMaintainingRedundancy parameters have a CurrentState value of Active. The Primary Activation Manager (PAM) considers the server availability critical to provide redundant availability of the database copies hosted by this server. 

Each of state-parameter shows three pieces of information:

• CurrentState
  The current state, either Active  or Inactive
   
• LastActiveTransition
  The timestamp of the last state change to Active
   
• LastInactiveTransition
  The timestamp of the last state change to Inactive

But there is still the bothering question of why are two of the six servers not safe for activating maintenance?

The reason is simple. The mailbox databases mounted by the member servers of the DAG have a different number of database copies. This configuration is due to data storage capacity constraints.

The mailbox databases storing primary user mailboxes use four database copies per database. Those copies are evenly distributed across all six mailbox servers. Mailbox database storing online archive mailboxes use three copies per database. This database copy layout allows for safely activating server maintenance for one server at a time without risk to database redundancy.

The servers LOCEXS01 and LOCEXS04 hold mailbox databases with just two copies per configured database. Placing one of those two servers into maintenance mode reduces the database availability for these mailbox databases to one. Therefore, PAM informs us that database redundancy is at risk when activating maintenance for those two servers. 

Example - During Maintenance

This example shows the member server redundancy state while LOCEXS06 is in maintenance. The reason for monthly maintenance for installing Windows updates.

Maintenance was activated using the StartDagServerMaintenance.ps1 PowerShell script.

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup indag01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS02        True          False       Normal        True               17.02.2020 11:04:12
LOCEXS03        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS06        True          True        High          True               17.02.2020 11:04:12
LOCEXS05        True          False       Prohibited    False              17.02.2020 11:04:12
LOCEXS04        True          False       Prohibited    False              17.02.2020 11:04:12

Having a single server in maintenance has a significant impact on all other servers in the DAG. The servers LOCEXS03 and LOCEXS05 are not safe for maintenance as well. Activating maintenance for those two servers would affect the database redundancy for the databases hosted by those two servers.

Example - After Maintenance

After completing all maintenance tasks, e.g., installing Windows Updates or a new Exchange Server Cumulative Update, you end server maintenance using the PowerShell script StopDagServerMaintenance.ps1.

We query the server redundancy state again. 

Get-MailboxServerRedundancy -DatabaseAvailabilityGroup indag01

Identity        IsServerFound IsInMainten RepairUrgency SafeForMaintenance HealthInfoLastUpdateTime
                InAD          ance
--------        ------------- ----------- ------------- ------------------ ------------------------
LOCEXS01        True          False       Prohibited    False              17.02.2020 11:23:12
LOCEXS02        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS03        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS06        True          False       High          True               17.02.2020 11:23:12
LOCEXS05        True          False       Normal        True               17.02.2020 11:23:12
LOCEXS04        True          False       Prohibited    False              17.02.2020 11:23:12

Server LOCEXS06 is not in maintenance, but the RepairUrgency state is High. The local Exchange Server replication engine is still busy replicating and processing log files, and updating the search indices. When CopyQueueLength and ReplayQueueLength are back to 0, and ContentIndexStates are back to Healthy, the RepairUrgency switches to Normal.

Tip

• You receive an error message when activating maintenance for an Exchange Server not safe for maintenance using
  ​StartDagServerMaintenance.ps1 -serverName [SERVER]
  
  In this case, you must use:

.\StartDagServerMaintenance.ps1 -serverName SERVERNAME -overrideMinimumTwoCopies:$true

Enjoy Exchange Server!

Azure AD Pass-through authentication (PTA) recommends that you run at least three authentication agents to provide high availability for authentication. 

When you download and install the PTA agent, registering the PTA agent to Azure AD might fail. This happens most of the time when the network connectivity to Azure AD requires the use of a proxy server. In such a network setup you normally encounter configuration errors only, if the proxy server is misconfigured or the Internet Explorer zone configuration is missing required entries for trusted sites.

When you encounter an error during installation and registration of the dedicated PTA agent I recommend to separate these two steps. You need the credentials of an Azure AD account that is a member of the Global Administrator management group.

1. Download the most current release of the PTA agent: https://aka.ms/getauthagent
2. Copy the downloaded file to the server that will serve as a PTA agent
3. Open an administrative command prompt and install the PTA agent software in silent mode without registering the agent:

AADConnectAuthAgentSetup.exe REGISTERCONNECTOR="false" /q

4. Open an administrative PowerShell session, navigate to the default installation location and register the PTA agent manually

# navigate to the default installation location
cd "C:\Program Files\Microsoft Azure AD Connect Authentication Agent"

# enter the global admin credentials
$cred = Get-Credential

# register the PTA agent using the RegisterConnector.ps1 script
# multiline example
.\RegisterConnector.ps1 `
-ModulePath "C:\Program Files\Microsoft Azure AD Connect Authentication Agent\Modules\" `
-ModuleName "PassthroughAuthPSModule" `
-AuthenticationMode Credentials ` 
-UserCredentials $cred `
-Feature PassthroughAuthentication

# single line example
.\RegisterConnector.ps1 -ModulePath "C:\Program Files\Microsoft Azure AD Connect Authentication Agent\Modules\" -ModuleName "PassthroughAuthPSModule" -AuthenticationMode Credentials -UserCredentials $cred -Feature PassthroughAuthentication

The Azure AD Pass-through agent Quickstart documentation has an example for automating the installation of the PTA agent as part of a server provisioning process. The current example references the wrong PowerShell module named AppProxyPSModule. The most recent release of the PTA agent does not contain a PowerShell module by that name. Use the PowerShell module PassthroughAuthPSModule, as shown in the PowerShell example shown above.

Links

• Azure Active Directory Pass-through Authentication: Quickstart
• Hybrid Identity Required Ports and Protocols

Enjoy Azure AD!