Just can't get enough of IT

When you change AutoDiscover settings for users, it can take up to 2 hours until the cached data is invalidated and the new AutoD configuration is sent as a response to new AutoD request.

You have to force a service and a application pool restart to activate your configuration changes immediately:

• Service: MSExchangeServiceHost
• Application Pool: MSExchangeAutodiscoverAppPool

These restarts need to be performed on each Exchange 2013/2016 server in your infrastructure serving AutoDiscover requests.

Use the following two PowerShell cmdlets to simplify this task:

Get-ExchangeServer | ? { $_.AdminDisplayVersion -like '*15.*'} | % { Invoke-Command -ComputerName $_.Name -ScriptBlock {Restart-WebAppPool MSExchangeAutodiscoverAppPool } }

Get-ExchangeServer | ? { $_.AdminDisplayVersion -like '*15.*'} | % { Invoke-Command -ComputerName $_.Name -ScriptBlock {Restart-Service MSExchangeServiceHost } }

Enjoy

The JET Blue Story

The Exchange Server product used the Extensible Storage Engine (ESE, aka JET Blue) to store data for decades. The story of the JET Blue (in contrast to JET Red which is used for Access database) can be read here (https://en.wikipedia.org/wiki/Microsoft_Jet_Database_Engine). In the acient days of data storage the ESE database was the best choice for storing mostly unstructured data with many dynamic properties.

The Messaging API (MAPI) had been developed in the 1990s to provide programmers with a set of unified interface for easier message exchange. The MAPI documentation at TechNet has been replaced by the current Outlook 2013 MAPI Reference. In todays world it is not easy to find reliable ressource about the original MAPI implementation. The only printed resource is Inside Mapi (Microsoft Programming Series) , ISBN 978-1572313125 , which has been published in 1996.

At Ignite 2015 Ross Smith VI joked about moving the Exchange storage engine to SQL. Back in the day with Exchange 2013 in production and Exchange 2016 coming, this was true. But Ross laid the tracks for the evolution of Exchange.

But it seems that the Exchange Product Team realized that in today's world with heavily standardized communication and less dynamic requirements than in the 1990s the days of JET blue are over. At the same time SQL Server evolved to mature database solution, capable of handling big data. The question was, if it can store SharePoint data, why not Exchange data. After twenty years of Exchange Server using the good ole ESE engine it was time to move on.

The SQL scripts that are used by Exchange to configure SQL are loacted in $exbin\SQL

Example

CREATE TABLE [dbo].[MAPI_PROPERTIES](
	[MAPI_PROPERTTY_ID] [int] IDENTITY(1,1) NOT NULL,
	[MAPI_PROPERTY_NAME] [nchar](127) NOT NULL,
	[IsWellKnownProperty] [bit] NOT NULL,
	[MAPI_TYPE_ID] [int] NOT NULL,
 CONSTRAINT [PK_MAPI_PROPERTIES] PRIMARY KEY CLUSTERED 
(
	[MAPI_PROPERTTY_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

ALTER TABLE [dbo].[MAPI_PROPERTIES]  WITH CHECK ADD  CONSTRAINT [FK_MAPI_PROPERTIES_MAPI_TYPES] FOREIGN KEY([MAPI_TYPE_ID])
REFERENCES [dbo].[MAPI_TYPES] ([MAPI_TYPE_ID])
GO

ALTER TABLE [dbo].[MAPI_PROPERTIES] CHECK CONSTRAINT [FK_MAPI_PROPERTIES_MAPI_TYPES]
GO

Exchange Server goes SQL

The current Exchange 2016 CU2 Preview supports an undocumented registry key to activate SQL Server support for Exchange. Personally I do not know, if this was supposed to be officially included in a public realease. So maybe the SQL support was made available by error and is already removed from the most current build again.

The famous SqueakyLobster registry key in has been used in Exchange 5.5 to troubleshoot performance issues. The new "Lobster" key is used to activate hidden code in Exchange Server product. The name of the key is LobsterMapiDB.

This key activates support for Exchange modern storage. Without this key you won't be able to move mailboxes from ESE legacy storage to SQL modern storage.

It is assumed that a SQL Server 2014 instance is available. A SQL Server 2014 Express edition is sufficient for testing purposes.

Note:
Any changes to configurations or the registry should be validated in a test environment first. Never try this in production right away.

The high level steps required to activate SQL support for Exchange 2016 are:

• Create a configuration file to provide the SQL connection string
• Create a SQL server login for the Exchange Trusted Subsystem security group
• Add a registry key to the local Exchange Server registry
• Restart Information Store Service (MSExchangeIS)
• Execute PowerShell script to migrate mailboxes to SQL

The detailed steps are:

• Create a new config file named Microsoft.Exchange.Data.SQL.exe.config in $exinstall/bin

		<?xml version="1.0" encoding="utf-8" ?>
		<!-- Exchange SQL Configuration - preliminary support -->
		<!-- %MAILBOXDATABASENAME% will be replaced by Exchange -->
		<!-- More information https://goo.gl/QiTtDo -->
		<configuration>
			<sectionGroup name="SqlMapiProviderGroup" type="Microsoft.Exchange.Data.SQL.SqlMapiProviderGroup, Microsoft.Exchange.Data.Common, Version=15.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
				<section name="SqlMapiProviderSection" type="Microsoft.Exchange.Data.SQL.SqlMapiProviderGroup, Microsoft.Exchange.Data.Common, Version=15.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
		    </sectionGroup>
			<runtime>
				<gcServer enabled="True" />
				<generatePublisherEvidence="False" />
			</runtime>
			<appSettings>
				<add key="MigrateMailboxesAutomatically" value="false" /> <!-- Not yet supported -->
				<add key="AllowJETBlueCoexistence" value="true" /> <!-- Allows for SQL/ESE Coexistence in DAG -->
				<add key="PerDatabaseMaxSize" value="1GB" />
				<add key="VerboseLoggingEnabled" value="False" />
		
			</appSettings>
			<SqlMapiProviderSection>
				<SqlMapiProvider>
					<add name="LobsterMapiDB"
					providerName="System.Data.SqlClient"
					connectionString="Data Source=SERVERNAME\INSTANCE;Initial Catalog=%MAILBOXDATABASENAME%;Integrated Security=True;MultipleActiveResultSets=True" />
				</SqlMapiProvider>
			</SqlMapiProviderSection>
		</configuration>

• Create a SQL login for Exchange Trusted Subsystem

CREATE LOGIN [DOMAIN\Exchange Trusted Subsystem] FROM WINDOWS

• Create a new DWORD named LobsterMapiDB in
  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15
  and set the data value to 1
  
  
• Restart Exchange Information Store
  
  Restart-Service MSExchangeIS
   
• Use Move-MailboxToModernStorage.ps1 script to move selected mailboxes to modern storage

More can be found here:

• Migration scripts and config files are available at Github, https://github.com/Apoc70/LobsterMapiDB

Links

• From crush to product documentation: The story of Squeaky Lobster, http://blogs.technet.com/b/exchange/archive/2006/06/15/3394229.aspx
• Setting "Squeaky Lobster" performance counter settings, https://support.microsoft.com/en-us/kb/556030
• Extensible Storage Engine, https://en.wikipedia.org/wiki/Extensible_Storage_Engine
• Outlook 2013 MAPI Reference, https://msdn.microsoft.com/en-us/library/cc765775.aspx
• SQL Server 2014, https://www.microsoft.com/en/server-cloud/products/sql-server/
• Final information: https://goo.gl/QiTtDo

Enjoy Exchange for the next 20 years...

There are three different ways to configure new Exchange user mailboxes after these have been created.

• The classic manual administrator approach (keeps your job safe, but is it fun?)
• The workflow based approach using some kind of IDM workflow solution (keeps the IDM consultant's job safe)
• The scripting agent approach by extending the Exchange cmdlets (keeps your job safe, is fun, keeps you in control and get's you more free time)

The Exchange cmdlet extension is controlled by a scripting agent configuration file and a organizational setting to enable/disable the scripting agent.

Preparation

A scripting agent configuration file sample (ScriptingAgentConfig.xml.sample) is located in

• $exinstall\Bin\CmdletExtensionAgents

The sample needs to be renamed to ScriptingAgentConfig.xml, to be picked up the PowerShell engine.

As always, a slight reminder: Test any modification in a test environment first, before you use the extension in a production environment.

After succesfull testing and deployment, you need to enable the scripting agent using

Enable-CmdletExtensionAgent "Scripting Agent"

Example

Even thought that you can extend mostly any Exchange cmdlet, this example covers the extension of the New-Mailbox and Enable-Mailbox cmdlets in a multi domain and multi AD site environment.

This extension disables the following CAS mailbox settings, after a new mailbox has been created:

• ActiveSync
• IMAP4
• POP3
• MAPI over HTTP

What does the example do?

• Extension is named MailboxProvisioning and handles the cmdlets New-Mailbox and Enable-Mailbox
• Is called on trigger OnComplete
  • The extension code is called after the original cmdlet has finished
• Code is executed, if the original cmdlet was successfully finished
• Code is executed, if the mailbox created is not an archive
• A slight delay of 10 seconds ensures that domain controller activities have been finished
  • Can be adjusted or even removed, depending on your environment
• Try to fetch at least on of three user parameters to identify the user mailbox
  • Checking for Identity, Name, Alias
• Fetch a list of all domain controllers in the current AD site where the Exchange server is located
• Iterate through the list of domain controllers and try to fetch the new CAS mailbox
  • If fetched, remember the domain controller's FQDN
• Change the CAS mailbox settings as needed and use the remembered domain controller as DC to write to

<?xml version="1.0" encoding="utf-8" ?>
  <Configuration version="1.0">
	<Feature Name="MailboxProvisioning" Cmdlets="New-Mailbox,Enable-Mailbox">
		<ApiCall Name="OnComplete">
			If ($succeeded) {
				if (!($provisioningHandler.UserSpecifiedParameters.Archive -eq $true)) {
					# delay execution for 10 seconds, adjust as needed
					Start-Sleep -s 10
					
					# validate parameters to use a not null parameter
					if ($provisioningHandler.UserSpecifiedParameters["Identity"] -ne $null) {
						$user = $provisioningHandler.UserSpecifiedParameters["Identity"].ToString()
					}
					elseif ($provisioningHandler.UserSpecifiedParameters["Name"] -ne $null) {
						$user = $provisioningHandler.UserSpecifiedParameters["Name"].ToString()
					}
					else {
						$user = $provisioningHandler.UserSpecifiedParameters["Alias"].ToString()
					}
						
					# view entire forest in a multi domain environment
					Set-AdServerSettings -ViewEntireForest:$true
					
					# fetch domain controllers in AD site}
					$server = Get-ExchangeServer $env:computername
					$DCs = Get-DomainController | ?{$_.adsite -eq $server.site}
					
					$CasMailbox = $null
					foreach($d in $DCs) {
						while($CasMailbox -eq $null) {
							# find a valid domain controller having the updated user object
							$CasMailbox = Get-CASMailbox $user -DomainController $d.dnshostname -ErrorAction SilentlyContinue
							# fetch DCs FQDN
							$WriteDC = $d.DnsHostName
							break
						}
					}
					
					try {
						# set CAS features as needed
						Set-CasMailbox $user -ActiveSyncEnabled:$false -ImapEnabled:$false -PopEnabled:$false -MapiHttpEnabled:$false -DomainController $WriteDC -ErrorAction SilentlyContinue
					}
					catch {}
				}
			}
		</ApiCall>
	</Feature>
</Configuration>

Notes

After adding the PowerShell code to the ScriptingAgentConfig.xml file, the file needs to be distributed across all Exchange servers. For distribution of the scripting agent configuration file I personally recommend Paul Cunningham's PowerShell script.

Be aware of the fact, that the scripting agent Xml is being validated using a strict schema validation. The scripting agent Xml is case sensitive, as noted here.

Links

• PowerShell Script to Distribute Scripting Agent Configuration File to Exchange Servers
• Cmdlet Extension Agents & XML Case Sensitivity
• Exchange Scripting Agent – The Power Of Script

Michael Hall of MCS has published on excellent Excel tool to simplify the migration of on-premise Exchange mailboxes to Office 365.

The script does support not special characters in display names as it is intended to work with ASCII files.

The textfile type used by Excel VBA code has been changed from 437 to 65001.

When you want to use the attached Excel file you are required to create an UTF-8 export when collecting mailbox data. The required PowerShell code is as follows:

$mbx=Get-Mailbox -resultsize unlimited; $mbx | foreach-object {$UPN = $_.UserPrincipalName; $EmailAddress = $_.PrimarySmtpAddress;$OU = $_.OrganizationalUnit; $Type = $_.RecipientTypeDetails; $_ | Get-MailboxStatistics | select @{Name="UPN";expression={$UPN}},@{Name="EmailAddress";expression={$EmailAddress}},@{Name="Type";expression={$Type}},@{Name="OU";expression={$OU}},DisplayName,@{Name="TotalItemSize(MB)";expression={$_.TotalItemSize.Value.ToMB()}},LastLogonTime}|Export-csv .\Mailboxes_Output.csv -notype	-Encoding UTF8

Then just follow the instructions provided in Michaels blog post.

Links

• Original Blog Post
• TechNet Gallery Post
• Modified Excel Spreadsheet at Github

Download

• Office365_Hybrid_Migration_Management_1.2 - UTF8.zip

Unsure, if you should migrate to Office 365? You want to know more about security of cloud applications and services? Your Exchange Server infrastructure requires an upgrade? Contact me via email: thomas@mcsmemail.de

Problem

It might happen that a mobile device running an Android operating system is not being redirected properly by the on-premises AutoDiscover service, when the mailbox has been migrated to Office 365.

If your device is not redirected, the device prefix is not recognized by Exchange Server and therefore not being redirected properly. The new device redirect feature for Android devices was introduced in Exchange Server 2010 SP3 RU9, Exchange Server 2013 CU8, and Exchange Server 2016.

The following device prefixes are known to Exchange by default:

• Acer, ADR9, Ally, Amazon, Android, ASUS, EasClient, FUJITSU, HTC, HUAWEI, LG, LS, Moto, Mozilla, NEC, Nokia, Palm, PANASONIC, PANTECH, Remoba, Samsung, SEMC, SHARP, SONY-, TOSHIBA, Vortex, VS, ZTE

Solution

If the device prefix of your device is not part of the default list, you can add the prefix to the AutoDiscover web.config file. 

Add the device prefix to the MobileSyncRedirectBypassClientPrefixes key in the appSettings node.

  <appSettings>
    <add key="LiveIdBasicAuthModule.AllowLiveIDOnlyAuth" value="true" />
    <add key="LiveIdBasicAuthModule.ApplicationName" value="Microsoft.Exchange.Autodiscover" />
    <add key="LiveIdBasicAuthModule.RecoverableErrorStatus" value="456" />
    <add key="LiveIdBasicAuthModule.PasswordExpiredErrorStatus" value="457" />
    <add key="ActiveManagerCacheExpirationIntervalSecs" value="5" />
    <add key="ProxyRequestTimeOutInMilliSeconds" value="30000" />
    <add key="LiveIdNegotiateAuxiliaryModule.AllowLiveIDOnlyAuth" value="true" />
    <add key="TrustedClientsForInstanceBasedPerfCounters" value="bes" />
    <add key="InstanceBasedPerfCounterTimeWindowInterval" value="900000" />
    <add key="MobileSyncRedirectBypassEnabled" value="true" />
    <add key="MobileSyncRedirectBypassClientPrefixes" value="Acer,ADR9,Ally,Amazon,Android,ASUS,EasClient,FUJITSU,HTC,HUAWEI,LG,LS,Moto,Mozilla,NEC,Nokia,Palm,PANASONIC,PANTECH,Remoba,Samsung,SEMC,SHARP,SONY-,TOSHIBA,Vortex,VS,ZTE" />
  </appSettings>

File location

%ExchangeInstallPath%\ClientAccess\Autodiscover\web.config

Notes

• Modify the web.config on each Exchange 2010/2013 Client Access Server and each Exchange 2016 server.
• After installing an Exchange 2013/2016 CU, the web.config must be modified again.

As always: Be careful when modifying application settings. Test such changes in a test environment first, if possible.

Links

• Some Android devices do not redirect to Exchange Online
• Exchange ActiveSync on-boarding to Office 365
• Exchange HTTP ActiveSync Protocol: HTTP Error 451

You need assistance with your Exchange Server setup? You have questions about your Exchange Server infrastructure and going hybrid? You are interested in what Exchange Server 2016 has to offer for your environment?

Contact me at thomas@mcsmemail.de
Follow at https://twitter.com/stensitzki

Description

The SQL Backup Helper is a set of Stored Procedures and User Defined Functions, which help you automate the process of creating backup sets, cloning database and creating a history set of database backups when using a SQL Server or SQL Server Express editions.

Works with

• SQL Server 2005 or higher
• SQL Server 2005 Express or higher

Features

• Create full backups of selected databases, using wildcards of inclusion and exclusion
• Create differential backups of selected databases, using wildcards of inclusion and exclusion
• Create transaction log backups of selected databases, using wildcards of inclusion and exclusion
• Clone databases
• Copy backup sets to local or UNC folders to create a backup history

Requirements

• xp_cmdshell
  The extended SP xp_cmdshell must be enabled, as the SPs need access to the file system.
• Access Rights
  The SQL Server Agent service account (when using SQL Server full edition) needs modify rights (at least) on the configured folders.

Installation

The SQL Backup Helper can be installed by using the setup binary (Red Gates SQL packager executable). The binary lets you choose the following:

• Database name
• Database folder
• Log folder
• Collation
• Recovery model
• Compatibility level
• Initial database size

The SQL Backup Helper can be installed by setting up the database manually and by running the Schema and MasterData scripts.

Examples

-- How to use SQL Backup Helper
-- Example FULL backup command performing a full backup of all databases, 
-- excluding master,model,msdb,tempdb databases 

EXEC USP_BackupDatabase 'FULL', NULL, 'master,model,msdb,tempdb'


-- Example FULL backup command performing a FULL backup of all databases 
-- starting with DEV, excluding master,model,msdb,tempdb databases

EXEC USP_BackupDatabase 'FULL', 'DEV%', 'master,model,msdb,tempdb'


-- Example for cleaning up history, keeping 3 backup sets, having no maximum age, 
-- including all databases, excluding master,model,msdb,tempdb databases

EXEC USP_CleanUpHistory 3, NULL, NULL, 'master,model,msdb,tempdb'


-- Example DIFF backup command performing a differential backup of all databases, 
-- excluding master,model,msdb,tempdb databases 

EXEC USP_BackupDatabase 'DIFF', NULL, 'master,model,msdb,tempdb'


-- Example TRAN backup command performing a tranaction log backup of all databases, 
-- excluding master,model,msdb,tempdb databases 

EXEC USP_BackupDatabase 'TRAN', NULL, 'master,model,msdb,tempdb'

Version History

• 1.0, Initial community release

Links

• Download and follow at Github: https://github.com/Apoc70/SQLBackupHelper
• Download and vote at TechNet Gallery: https://gallery.technet.microsoft.com/Simplify-SQL-Backup-Jobs-59a71c00

Additional Credits

Additional credits go to Markus Heiliger for the initial version of the SQL Backup Helper.

Follow

• Twitter @stensitzki
• LinkedIn
• Xing