This article came out in February and I have been behind on my blog updates due to my current project, but I feel this post is important and am going to relay the message that I received here for your review. Thanks again for your support of this blog and its continued longevity.
Microsoft periodically refreshes certificates in Office 365 as part of our effort to maintain a highly available and secure environment. From Jan 23rd, 2021, we are making a certificate change on our Microsoft Federation Gateway every six weeks that could affect some customers as detailed in this knowledge base article. Please note that longer term, this “six week” rhythm to renew the certificate will be further shortened to daily renewals which will further enhance security of the environment. The good news is you can easily avoid any disruption.
Who is affected?
This certificate change can affect any customer that is using the Microsoft Federation Gateway (MFG). If you are in a hybrid configuration that relies on a Federation Trust established with MFG in the Exchange on-premises organization or if you are sharing free/busy information between two different on-premises organizations using the Microsoft Federation Gateway as a trust broker, you need to take action.
When will the change occur?
The change is scheduled to occur every six weeks to begin with, with this frequency further increasing. You must take action to avoid any disruptions.
What type of issues will you face if no action is taken?
If you don’t take action, you won’t be able to use services that rely on the Microsoft Federation Gateway. For example:
A cloud user might not be able to see free/busy information for an on-premises user and vice versa.
MailTips might not work in a Hybrid configuration.
Cross-premises free/busy might stop working between organizations that have organization relationships in place.
Additionally, if you run the Test-FederationTrust cmdlet, you might receive an error message that indicates that the Delegation token has validation issues. For example, you receive an error message that resembles the following:
Id : TokenValidation Type : Error Message : Failed to validate delegation token.
And, you might receive one of the following error messages in the Exchange Web Services (EWS) responses:
An error occurred when processing the security tokens in the message Autodiscover failed for email address User@contoso.com with error System.Web.Services.Protocols.SoapHeaderException: An error occurred when verifying security for the message
What action should you take?
You can use the following command on your Exchange Server to create a scheduled task to run the update process daily. This is how we recommend you keep your Federation Trust constantly updated. This will prevent you from being negatively affected by future metadata changes.
If you prefer to not use a scheduled task, you can manually run the command at any time to refresh the metadata. This is not recommended as the frequency to refresh certificate will increase from 6 week period to daily, and manually updating this would be quite cumbersome.
Changes have been made to the HCW and the installation since this original post. Please read the following to gain knowledge of the updates to the tool and the installation.
We wanted to let you know that we are releasing what we consider a significant update to Exchange Hybrid Configuration Wizard (HCW). Along with a handful of small bug fixes, there are four major changes coming that we wanted to share with you:
HCW will no longer enable Federation Trust by default for all installations. Instead, it will only enable Federation Trust if there are Exchange 2010 servers on premises. HCW will call Get-ExchangeServer and if no Exchange 2010 servers are reported, the workflow to enable Federation Trust and subsequently require domain proof will not execute. Note that organization relationships are still created.
When uninstalling the hybrid agent and switching to Classic in the HCW, this action would sometimes fail with a “null reference” error. We have fixed this!
How many of you have hit the HCW 8064 error – unable to configure OAuth, and subsequently had no idea why OAuth failed to configure? Yes, we heard you loud and clear! In this release, we have completely changed the way we enable and configure OAuth. Instead of enabling OAuth at the service layer, we now enable OAuth via a Graph API under the context of the Tenant Admin. This in turn removes the error obfuscation we had with the service layer enablement and allows us to include a detailed error entry in the HCW log. So while you still see the HCW 8064 error in the HCW UI, you can now review the log for the specific error detail which will make it easier to troubleshoot and resolve.
When verifying DNS, we had a fallback mechanism that would reach out to an external site to verify domains. While this fallback mechanism was rarely hit, we received overwhelming feedback to not use this mechanism/site as it was not listed in our IPs & URLs web page. We have removed that fallback and now only use the endpoint “mshybridservice.trafficmanager.net”, which is listed in our endpoints documentation.
Because this is a major version update, the build begins with 17.x vs 16.x. The build number can be found in the top right corner once you download and open the HCW.
Because of the web-based distribution nature HCW uses and this version is a brand new package, you will get all this goodness simply by installing the new HCW from here. The current builds of HCW (16.x) will not automatically update to 17.x build, in fact – you could run the two side-by-side. Once you are on 17.x build – the HCW will then auto-update as usual.
A few additional notes: At this time, we do not anticipate new HCW 16.x builds. Therefore, to continue getting new HCW builds in the future, uninstall the current version of HCW (16.x) and then install the new version (17.x). The new version of HCW has a new dependency, .NET 4.7.2. The installer should take care of this for you, but just so you are aware.
ORIGINAL POST
I’m working on getting certified in Exchange Hybrid Scenarios and Exchange Online configuration as part of my skill set for Exchange. In doing so, I had successfully implemented a complete Full Hybrid Exchange Environment between my Exchange Online Tenant and my On Premises Exchange 2019 Environment last evening.
I wanted to give an update that was posted to my LinkedIn Posting on this. Thank you Brian Day for the vote of confidence and caution that running these cmdlets manually is not supported by Microsoft and that the HCW, like all the Online Microsoft Products, is constantly changing and being updated.
Important Note
As preparation, I bought some Exchange Online Plan 1 licenses which give me a 50 GB mailbox limit and basic mailbox functionality. It does not include the more advanced features such as ATP, or DLP. I am running most of those features through my On Premises Environment. I mainly wanted to be able to place mailboxes in the cloud and have a hybrid setup. My plan was to have mail flow continue through my On Premises environment so that my Exchange Server features would be used and I would not have to change any MX or SPF records. I also had my certificates in place for SSL and OWA so I would want keep mail flow routed that way, through on premises. I do want to be able to have Free/Busy lookups cross-premise so federation would have to be enabled as well. I would also have to enable the MRS proxy on my Exchange Server so that mailbox migration could be implemented cross-premise. I also have previously configured Azure AD Sync along with ADFS for Single Sign On. In my case, another server was not needed as I didn’t have enough mailboxes or real need to split my frontend and backend deployment. Running the Hybrid Configuration Wizard would not open any new ports or change any existing port traffic that was already configured on my firewall. These are just a few of the considerations that need to be looked at when considering a hybrid integration.
So, once I had all those considerations handled in my design, I ran the Hybrid Configuration Wizard. What I want to do in this blog post is to go through the steps that the wizard does in the background to setup the Hybrid Environment as you go through the Wizard.
I mainly used the following blog post as a reference, but have approached it differently by diving into the cmdlets that are run during the process:
1. The HCW validates the On-premises and Online Exchange Connection.
The Hybrid Configuration Wizard checks if it is possible to connect to both servers with PowerShell. It runs the Get-ExchangeServer cmdlet on premises after resolving the server in DNS. It then connects to Exchange Online, authorizing the connection:
3. The HCW collects information on the Exchange online (Office 365) configuration
This task repeats what has been done in the previous step, only for the Exchange online, instead of the on-premises one.
The cmdlets include, in order:
1
2
3
4
5
6
7
Get-OrganizationConfig
Get-OnPremisesOrganization
Get-AcceptedDomain
Get-MigrationEndpoint
4. Federation Trust is determined. If not present, a new Federation Trust and the required certificate will be created on the local Exchange Server
You will be prompted in the Wizard to create a Federation Trust if not present. The following articles explain Federation and its requirements:
Understanding Federation – Link Here Understanding Federated Delegation – Link Here Create a Federation Trust – Link Here
If the activity is finished successfully, a new certificate should appear on the on-premises Exchange Certificates list. The new certificate includes “Federation” in its Subject field. To make sure the certificate is there, you can run a cmdlet: Get-ExchangeCertificate | ft -a -wr
The results will look like this
5. The HCW createsa new Hybrid Configuration Object in the local Active Directory
The HCW will run cmdlets based on the information you provide in the HCW for the certificate, the on premises Exchange Server, the domain(s), and what features you want turned on:
7. The HCW Configures the Organization Relationship between the local server and the cloud.
This configuration is not necessary in minimal hybrid deployment. Since I have a full hybrid deployment configured, the cmdlets were run as needed to configure it. Thanks to the correct configuration, it is possible to synchronize free/busy status of mailboxes and their elements between the on-premises Exchange Environment and Exchange online.
New-OrganizationRelationship-Name'On-premises to O365 - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-TargetApplicationUri'outlook.com'-TargetAutodiscoverEpr'https://autodiscover-s.outlook.com/autodiscover/autodiscover.svc/WSSecurity'-Enabled:$true-DomainNames'LDLNET.mail.onmicrosoft.com'
New-OrganizationRelationship-Name'O365 to On-premises - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-TargetApplicationUri'FYDIBOHF25SPDLT.ldlnet.net'-TargetAutodiscoverEpr'https://autodiscover.ldlnet.net/autodiscover/autodiscover.svc/WSSecurity'-Enabled:$true-DomainNames'ldlnet.net'
Set-OrganizationRelationship-MailboxMoveEnabled:$true-FreeBusyAccessEnabled:$true-FreeBusyAccessLevel LimitedDetails-ArchiveAccessEnabled:$true-MailTipsAccessEnabled:$true-MailTipsAccessLevel All-DeliveryReportEnabled:$true-PhotosEnabled:$true-TargetOwaURL'http://outlook.com/owa/LDLNET.onmicrosoft.com'-Identity'On-premises to O365 - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'
Set-OrganizationRelationship-FreeBusyAccessEnabled:$true-FreeBusyAccessLevel LimitedDetails-MailTipsAccessEnabled:$true-MailTipsAccessLevel All-DeliveryReportEnabled:$true-PhotosEnabled:$true-TargetOwaURL'https://mail.ldlnet.net/owa'-Identity'O365 to On-premises - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'
8. The HCW and setting connectors on both Exchange servers
The HCW checks to see if the connectors are there, if not, it sets them up. During this workflow, four connectors are set – one receive and one send connector for each server. Those connectors guarantee the mail flow between the on-premises and Exchange Online.
New-InboundConnector-Name'Inbound from 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-CloudServicesMailEnabled:$true-ConnectorSource HybridWizard-ConnectorType OnPremises-RequireTLS:$true-SenderDomains''-SenderIPAddresses$null-RestrictDomainsToIPAddresses:$false-TLSSenderCertificateName'.ldlnet.net'-AssociatedAcceptedDomains$null
New-OutboundConnector-Name'Outbound to 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-RecipientDomains'*'-SmartHosts'mail.ldlnet.net'-ConnectorSource HybridWizard-ConnectorType OnPremises-TLSSettings DomainValidation-TLSDomain'mail.ldlnet.net'-CloudServicesMailEnabled:$true-RouteAllMessagesViaOnPremises:$true-UseMxRecord:$false-IsTransportRuleScoped:$false
9. The HCW configures OAuth Authentication across the Hybrid
This LINKexplains how OAuth is configured between Exchange On Premises and Exchange Online. It’s a very good article to read as it shows how to get the Modern Authentication style working. Now the HCW does this for you and at the end of the article, you can run cmdlets to test the validity of the configuration.
Again, look at both of those links to get a little more detail as to what each cmdlet does and how it sets up OAuth. Here are the two cmdlets used to test OAuth:
Run On Premises PowerShell to test OAuth to Exchange Online
In order to be able to move mailboxes between Exchange On Premises and Exchange Online, you have to enable the Exchange Web Services Virtual Directory to use the MRSProxy (Microsoft Replication Service proxy). You also have to set your EWS Virtual Directory to use Basic Authentication. You’ll want to do this before running the HCW or else you will receive the following error when the HCW validates the Migration setup and configuration:
Microsoft.Exchange.Migration.MigrationServerConnectionFailedException: The connection to the server ‘mail.ldlnet.net’ could not be completed. —> Microsoft.Exchange.MailboxReplicationService.RemoteTransientException: The call to ‘https://mail.ldlnet.net/EWS/mrsproxy.svc’ failed. Error details: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. –> The remote server returned an error: (403) Forbidden.. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The remote server returned an error: (403) Forbidden.
Some of the cmdlets run to test Migration and MRS Proxy Settings are as follows:
The HCW runs from final cmdlets to finish up the installation of the Hybrid environment. Here are the cmdlets run:
1
2
3
Get-OnPremisesOrganization
Set-OnPremisesOrganization-Identity'48e7bec9-404c-4d24-b59e-4b46b64d7e03'-Comment'Bunch of letters and numbers'
All this information was found in the setup logs that are in the following directory C:\Users\%username%\AppData\Roaming\Microsoft\Exchange Hybrid Configuration
PLEASE LEAVE QUESTIONS, COMMENTS, UPDATES! I WOULD LOVE TO HEAR FROM YOU!
Cookies Notice for itblog.ldlnet.net
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. AcceptRead More
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
