In my experience with doing Exchange migrations to the cloud, there always seemed to be an issue with EWS throttling causing very slow mailbox moves. The remediation to this was to always contact Microsoft EXO Support and open a ticket to request that your EWS throttling for your tenant be lifted so you could move your mailboxes more quickly.
The reason for the EWS Throttling was to keep large amounts of data from flooding the front end server farm possibly causing a temporary outage or corruption of data going to the cloud since there are literally thousands of customers going through the same server farm and possibly migrations. Throttling was a way to keep everything in check.
Microsoft recently made an interesting change to the automated support handling capabilities of the Microsoft 365 admin center to handle requests for Exchange Web Services (EWS) throttling to be lifted for up to 90 days without human intervention.
Here is how to request the throttling be lifted via the M365 Support Assistant:
Go to the Help (?) section of the Microsoft 365 admin center.
Click the Need Help icon.
Enter “EWS throttling” as the search phrase.
Click Run tests when asked to check your environment. Essentially, the tests check what EWS throttling applies to the tenant.
Getting EWS Throttling Support
Running the Tests from the Admin Center
The support assistant checks the tenant settings and concludes that EWS is throttled (the normal situation). You’ll then be offered the chance to update the settings to the tenant EWS policy to lift throttling for 30, 60, or 90 days.
Select the number of days you’d like to adjust the policy for and then Update Settings.
After a short delay, the support assistant should confirm that the settings have been changed.
Select the number of days to lift throttling and click Update Settings
Once the setting is changed in the tenant, it will be effective after about 15 minutes for replication to the server farm. You should then be able to run your migrations at full speed.
Settings Changed Successfully
NOTE: Changing this setting is only effective for EWS migrations and NOT IMAP Migrations (such as from G-Mail).
Having this option online in the portal saves a tremendous amount of time when working to get your Exchange migrations to the cloud completed. Hopefully there will be more to come. I just hope it doesn’t keep me out of a job!
I AM CURRENTLY LOOKING FOR NEW OPPORTUNITES! POSITIVE MINDSET, POSITIVE ATTITUDE!
Since the COVID-19 virus has managed to get me laid-off and not working, I have not had too much to post in the past months. And although it has wavered greatly, I will remain positive and hope that someone will see the value that I can bring to their organization in the near future.
With that said, I wanted to repost this article that was sent to me as this issue arises quite often in the workplace when someone has left for vacation and is not in the office. So, here it is:
Out of Office replies can sometimes be a bit of a mystery to people; how do they work? What do you do if they don’t work? In this blog post, we will discuss the bits and pieces of Out of Office and some of the main reasons why an Out of Office (aka. OOF) reply might not get delivered to users. Note that while we are writing this from the viewpoint of an Exchange Online configuration, many of same things can be applied to on-premises configuration also. By the way – did you ever wonder why “OOF” is used instead of “OOO”? If you did, see this!
What is an Out of Office reply?
Out of Office replies are also known as OOF (or OOO) replies or automatic replies. They are Inbox rules that are set in the user’s mailbox by the client. OOF rules are server-side rules, so the response is sent regardless of whether a client is running, or not.
There are several ways of setting up automatic replies:
First, it can be set up as an automatic reply feature from Outlook, like this. It can also be configured using other clients, such as Outlook on the web (OWA), PowerShell command (Set-MailboxAutoReplyConfiguration). Admins can set up OOF replies on behalf of (forgetful) users from the M365 Admin Portal.
By design, Exchange Online Protection uses the high risk delivery pool (HRDP) to send the out of office replies, because they are lower priority messages.
Types of OOF rules
There are three types of OOF rules: Internal, External and Known Senders (Contact list). They are stored in the mailbox with the names in the following table. If a mailbox has set only Internal OOF, there will be no external rule created and the mailbox will have one OOF rule.
Note: Apart from OOF rule, other rules like the Junk Email rule will also have the “IPM.ExtendedRule.Message” message class; the MSG_NAME will determine what the rule is for.
Logon > profile that you are accessing > Top of information store > Inbox > right click ‘Open associated contents table’. They are listed under the Message Class column. All Inbox rules will have the same message class IPM.Rule.Version2.Message and there is one message class and name for each Inbox rule.
For all rules, the name of the rule is in stored in the PR_RULE_MSG_NAME property. So, if there are 4 Inbox rules, there will be 4 IPM.Rule.Version2.Message one for each rule, and the name of the rule is stored in PR_RULE_MSG_NAME.
OOF rules in MFCMapi:
And OOF rule templates in MFCMapi:
History of OOF replies
OOF response is sent once per recipient. Recipients to whom the OOF was sent are stored in the OOF history and are cleared out when the OOF state changes (enabled/disabled) or the OOF rule is modified. OOF history is stored in the user’s mailbox and can be seen using MFCMapi tool at: Freebusy Data > PR_DELEGATED_BY_RULE.
Note: If you want to send response to the sender every time instead of just once, you can apply mailbox server side rule “have server reply using a specific message” to send automatic reply instead of using the OOF rule. This server-side rule will send reply to the sender every time a message is received.
Now that we know what OOF replies are and how they are stored on the server, we can move on to address some of the scenarios where OOF is not sent to the sender. We will also discuss possible fixes and some more frequently seen issues you may have with OOF configuration.
The first category of issues we will talk about are issues related to OOF replies not being received by the sender of the original message.
OOF issues related to transport rules
When OOF doesn’t seem to be sent for all users in the tenant, usually there is a transport rule causing the issue. Check all the transport rules that may apply to the affected mailbox using step two of this article.
If you suspect a delivery problem, run a message trace from the Office 365 tenant. We know that OOF response is sent back to the original sender of the message, so for OOF messages, the sender of the original message becomes the recipient when tracking. We should then be able to tell if OOF reply has been triggered and sent to external or internal recipient. If a Transport rule is blocking the OOF response, the message trace will clearly show you that.
There is one scenario I would like to highlight when it comes to transport rules blocking OOF replies. Let’s assume that you moved the MX record to a 3rd party anti-spam solution; you have created a transport rule to reject any email coming from any other IP address than the 3rd party anti-spam.
The transport rule will look something like this:
Description: If the message: Is received from ‘Outside the organization’ Take the following actions: reject the message and include the explanation ‘You are not permitted to bypass the MX record!’ with the status code: ‘5.7.1’ Except if the message: sender ip addresses belong to one of these ranges: ‘1xx.1xx.7x.3x’
ManuallyModified: False
SenderAddressLocation: Envelope
As OOF replies have a blank (<>) Return-Path, you will see that the rule is unexpectedly matching the transport rule and the OOF responses are getting blocked.
In order to fix this, you can change the transport rule property of ‘Match sender address in message’ to ‘Header or envelope’, so the checks will also be done against ‘From’ (also known as the ‘Header From’ address), ‘Sender’ or ‘reply-to’ fields. More information about the mail flow rule conditions is here.
JournalingReportNdrTo mailbox setting
If the affected mailbox is the one which is configured under JournalingReportNdrTo, OOF replies will not be sent for that mailbox. Moreover, journaling emails may be affected as well. It is recommended to create a dedicated mailbox for the JournalingReportNdrTo setting. Alternatively, you can set it to an external address.
For more details on how to solve this, please see this KB Article.
Forwarding SMTP address is enabled on the mailbox
If the affected user mailbox has SMTP forwarding enabled, OOF replies won’t be generated. This can be checked in user mailbox settings (OWA):
You need pay attention to what OOF type you have set up, as this will impact the OOF response and OOF may not be generated at all if the configuration is incorrect. Let’s assume you have a hybrid organization with mailboxes hosted both in Exchange on-premises and Exchange Online. In this scenario, by design only external messages will be sent to on-premises while AllowedOOFType is set to External. To be able to send internal OOF messages to on-premises in hybrid environment, you need to set the AllowedOOFType to InternalLegacy.
You also have option to send external Out of Office replies only for contacts at the mailbox configuration level (ExternalAudience: Known). This can make automatic replies not being sent to anyone external but contacts. The command to check the configuration is:
Another setting on remote domains is one which lets you dictate whether you allow or prevent messages that are automatic replies from client email programs in your organization.
This can be found in Exchange Admin Center > Mail flow > Remote domains
Note: If this option is set to false, no automatic replies will be sent to users for that domain. This setting takes precedence over the automatic replies set up at the mailbox level or over the OOF type (discussed above).
Please, keep in mind that $false is the default value for new remote domains that you create as well as the built-in remote domain named Default in Exchange Online.
If the email was marked as spam and sent to junk, an automatic reply will not be generated at all.
Pretty self-explanatory, that one!
Message trace shows delivery failure
If you investigate an OOF reply issue and in the message trace you find the following error message:
“550 5.7.750 Service unavailable. Client blocked from sending from unregistered domains.”
You should reach out to Support to find out why the unregistered domain block was enforced.
There are some other scenarios that might come up when working with OOF replies, let’s cover those next!
An old or duplicate OOF message is sent
This is likely due to a duplicate Inbox rule or the OOF history limit. The OOF history has a limit of 10,000 entries, if this threshold is hit, OOF will continue to be sent to recipients that are not already in the list as any new users can’t be added to the list. All users already in the list will not receive duplicate OOF replies. For more information you may want to check this article or follow the action plan below.
Remove the OOF rules and the OOF rule templates from the mailbox. To locate the rules and delete them go back to > “Inbox OOF rules”
Disable and then re-enable the OOF feature for the mailbox
Now, you can check again whether the OOF feature works as expected and symptoms do not occur.
Automatic replies cannot be enabled; an error message is received
While attempting to access automatic replies from the Outlook client, an error message is received saying that “Your automatic reply settings cannot be displayed because the server is currently unavailable. Try again later.”
To narrow down this issue, you should perform the following steps:
Confirm EWS protocol is enabled on the mailbox as OOF replies rely on it to be enabled (note that re-enabling this might take several hours to take effect)
Enable the OOF feature by using the following command:Set-MailboxAutoReplyConfiguration <identity> -AutoReplyState Enabled
Check whether the OOF feature works as expected.
If the issue is still there, review the rules quota on the mailbox: Get-mailbox -identity <mailbox> | fl RulesQuota
By default, the RulesQuota has a maximum quota which is calculated by the size of the rules (not the number of rules). Maximum is 256 KB (262,144 bytes).
Remove the OOF rules and the OOF rule templates from the mailbox. To locate the rules and delete them go back to > “Inbox OOF rules” After you remove them, you can re-enable the OOF feature and then test again.
An automatic reply is still sent despite OOF being disabled
We have encountered scenarios in which OOF messages are still being sent, although it is disabled. Most of the time, we found that the rule is created manually by the end users using the out-of-office template.
So, as you can see, there is quite a bit that goes into troubleshooting OOF Replies and it is not all straight forward.
THANKS FOR READING! I’M AVAILABLE FOR WORK! PLEASE CONTACT ME FOR AN APPOINTMENT!
During my time as a PFE for Microsoft, I have encounted many issues with Federation in a Hybrid Exchange Deployment. Recently, the following support announcement came out and I wanted to share as I hope this can help others that may be having issues out there.
One of the more common causes of HCW failures is the Federation Trust step for the Exchange on-premises organizations in Full hybrid configurations (Classic or Modern topologies).
Federation trust is a mandatory step in the on-premises Exchange organizations when configuring Full hybrid deployments, as this allows us to create organization relationships (for features like hybrid free/busy or OWA/EAS redirection) and sharing policies (1:1 hybrid calendar sharing). In Exchange Online multi-tenant organizations, federation trust is already in place.
Below is an illustration of an Exchange hybrid deployment where both the Exchange on-premises organization and the Exchange Online organization have a trust with Azure Authentication System (formerly called Microsoft Federation Gateway):
Example of Hybrid Federation
Before getting to our subject, let’s quickly go over different hybrid configurations and Hybrid Configuration Wizard (HCW) – as this is the supported tool to configure hybrid deployments. There are 2 flavors of hybrid configurations: • Classic hybrid • Modern hybrid
At this time, each of those supports the following hybrid modes:
Full
Minimal (which further breaks down into…)
Express (a one-time sync)
“Actual minimal”
A quick overview of Full / Minimal / Express options, can be found here. More info on HCW is here.
As mentioned earlier, a federation trust is created by HCW only in Full Hybrid.
HCW logs are located at %appdata%\Microsoft\Exchange Hybrid Configuration on the machine from where HCW was ran. The easiest way to get to them is to press F12 in the HCW window to open the Diagnostic tools and from there you can Open Folder Logging or Open Log File directly.
When you have issues with federation trust, the log will usually show errors when one of the following cmdlets are executed:
Set-FederationOrganizationIdentifier or Add-FederatedDomain (but can be other cmdlets as well).
Once you identified the exact cmdlet failing and where (Session=OnPremises – means Exchange Management Shell and Session=Tenant means Exchange Online PowerShell), you should copy-paste the failing command and try to execute it manually and see if that is failing as well (most likely it will). You can also open the shells from F12 Diagnostic tools windows in HCW.
In order to get more details on the error and to rule out this is not an issue with HCW itself, you will need to separately run the same command that threw exception in HCW log and add Verbose switch to get verbose details of the error and the serialized remote exception.
For example, if the Exchange server version is Exchange 2010, you will run the failing command with Verbose switch in Exchange Management Shell (EMS), see if that fails and then get the serialized remote exception.
Once you’ve gathered the verbose error / serialized exception, try to understand where it is failing (or provide it to Microsoft Support together with the HCW log).
Common Errors with Remediation Steps
Federation trust fails with “Object reference not set to an instance of an object”
This is a known old issue on Exchange 2016 CU7 servers, make sure your Exchange servers are updated to the latest CU.
Full error in the HCW log:
Log Entries Extended
1
2
3
2017.10.0601:45:56.562*ERROR*10277[Client=UX,Activity=Domain Ownership,Session=OnPremises,Cmdlet=Set-FederatedOrganizationIdentifier,Thread=21]FINISH Time=398.4msResults=PowerShell failed toinvoke'Set-FederatedOrganizationIdentifier':Objectreference notset toan instance of an object.An unexpected error has occurred andaWatson dump isbeing generated:Objectreference notset toan instance of an object.
2017.10.0601:45:56.563*ERROR*10224[Client=UX,Page=DomainProof,Thread=21]Microsoft.Online.CSE.Hybrid.PowerShell.PowerShellInvokeException:PowerShell failed toinvoke'Set-FederatedOrganizationIdentifier':Objectreference notset toan instance of an object.An unexpected error has occurred andaWatson dump isbeing generated:Objectreference notset toan instance of an object.--->System.Management.Automation.RemoteException:Objectreference notset toan instance of an object.
2019.07.0617:53:15.37510177[Client=UX,Activity=Domain Ownership,Provider=OnPremises,Thread=19]PowerShell Error Record:{CategoryInfo={Activity=Add-FederatedDomain,Category=InvalidResult,Reason=DomainProofOwnershipException,TargetName=,TargetType=},ErrorDetails=,Exception=Proof of domain ownership has failed.Make sure that the TXT record forthe specified domain isavailable inDNS.The format of the TXT record should be"example.com IN TXT hash-value"where"example.com"isthe domain you want toconfigure forFederation and"hash-value"isthe proof value generated with"Get-FederatedDomainProof -DomainName example.com".,FullyQualifiedErrorId=367408EF,Microsoft.Exchange.Management.SystemConfigurationTasks.AddFederatedDomain}
Resolution:
• Check the TXT record for your domain(s) in HCW log or in Exchange Management Shell with command Get-FederatedDomainProof -DomainName • See if it matches your published TXT record with either nslookup utility or by checking internet websites like https://www.whatsmydns.net/ put your domain in hostnames, type=txt, Nameservers – Authoritative
You would look for errors, missing records or unusual formatting (characters, spaces, quotes, TXT record split in half).
Federation fails with “An unexpected error occurred on a receive” or “An unexpected error occurred on a send.”
Full error in the HCW log:
Log Entries Extended
1
2018.10.1017:03:31.277*ERROR*[Activity=Domain Ownership,Session=OnPremises,Cmdlet=Set-FederatedOrganizationIdentifier]FINISH Time=64.3sResults=PowerShell failed toinvoke'Set-FederatedOrganizationIdentifier':An error occurred whileattempting toprovision Exchange tothe Partner STS.Detailed Information"An error occurred accessing Windows Live. Detailed information: "The underlying connection was closed:An unexpected error occurred onareceive.".".
Resolution:
Check outbound access from all your Exchange Servers to Microsoft Federation Gateway by browsing using Internet Explorer with PSEXEC tool (with -s and -i switches) from the Exchange Server (this will use Internet Explorer under System Account / Exchange Server Account).
From on-premises Exchange to Office 365, the Exchange 2010 MBX & CAS or 2013 MBX (backend) or 2016 / 2019 would need outbound Internet access to the Microsoft Federation Gateway in addition to https://outlook.office365.com/ews/exchange.asmx
Verify the machine/system account can access these Microsoft Federation Gateway URLs:
Note: If the Exchange requires a proxy server to access the Internet, specify the proxy server using “Set-ExchangeServer myExchange01 -InternetWebProxy http://myproxy:80”. Notice such proxy can’t require any user authentication for outbound Internet access, and the proxy must start with HTTP: and not HTTPS: (secure SSL).
You can also set the proxy using netsh as well.
set proxy proxy-server=”http=myproxy;https=sproxy:88″ bypass-list=”*.contoso.com”
In rare instances, you can use the machine/system account to access the URLs from the browser, but Exchange cmdlets still failed with “Could not establish trust relationship for the SSL/TLS secure channel.” If that happens, make sure the certificate authorities for the urls are installed at the Third-Party Root Certification Authorities of the machine local certificate location.
Federated delegation features require that the Mailbox and Client Access servers in your organization have outbound access to the Internet by using HTTPS. You must allow outbound HTTPS access (port 443 for TCP) from all Exchange 2010 Mailbox and Client Access servers in the organization.
There is no specific error / exception, in HCW log you would see it stops without any specific error.
in HCW log if you see something with “DEL“: “contoso.com/Configuration/Deleted Objects/Microsoft Federation Gateway/DEL: <xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx>”.
Solution is to remove the orphaned federation trust and re-run HCW.
NOTE: as a first step, you can try to run the commandremove-federateddomain with the switch -Force. Also, you don’t need to recreate federation trust manually, just re-run HCW (this will recreate federation trust for us)
Federation Trust fails with “InternalError InternalError: Internal error.”.”.””
2019.08.2307:45:23.239*ERROR*10277[Client=UX,Activity=Domain Ownership,Session=OnPremises,Cmdlet=Set-FederatedOrganizationIdentifier,Thread=20]FINISH Time=325.0msResults=PowerShell failed toinvoke'Set-FederatedOrganizationIdentifier':An error occurred whileattempting toprovision Exchange tothe Partner STS.Detailed Information"An unexpected result was received from Windows Live. Detailed information: "InternalError InternalError:Internal error.".".{CategoryInfo={Activity=[System.String]Set-FederatedOrganizationIdentifier,Category=[System.Management.Automation.ErrorCategory]InvalidResult,Reason=[System.String]
Resolution:
Open request with Microsoft Support or check if any Service Incident is published. Please see this for more information.
Federation trust fails with “1007 Access Denied”
Full error in the HCW log:
Log Entries Extended
1
Set-FederatedOrganizationIdentifier,Category=[System.Management.Automation.ErrorCategory]InvalidResult,Reason=[System.String]ProvisioningFederatedExchangeException,TargetName=[System.String],TargetType=[System.String]},ErrorDetails=,Exception=[System.Management.Automation.RemoteException]An error occurred whileattempting toprovision Exchange tothe Partner STS.Detailed Information"An unexpected result was received from Windows Live. Detailed information: "1007AccessDenied:Access Denied."."
Resolution:
“1007 Access Denied” error is usually when we have issues with:
Outdated federation trust (for example, federation trust certificate expired) and in this case you would remove federation trust by following these steps.
If the federation trust certificate is not found on any of the servers, then proceed with resolution from the next error.
As an example, from one HCW log, there seems to be this federation trust certificate expired on 05/13/2019:
Federation trust fails with “Federation Certificate cannot be found”
Full error in the HCW log:
Log Entries Extended
1
2019.11.2214:40:32.569*ERROR*10277[Client=UX,Activity=Domain Ownership,Session=OnPremises,Cmdlet=Get-FederatedDomainProof,Thread=6]FINISH Time=125.1msResults=PowerShell failed toinvoke Get-FederatedDomainProof:Federation certificate with the thumbprint‘03650FFAF05E83E3B007DF3473CB5753F5C4459’cannot be found.
Resolution:
Follow the procedure here to manually cleanup the federation trust from AD. Once this is done, re-run the HCW to re-create it automatically.
KEEP TROUBLESHOOTING! REMAIN VIGILANT USING FOCUSED INTENT, NEVER EMOTIONALIZING (FINE)!
In a previous post, I showed how you could update one user’s photo for their Outlook and AD profiles via PowerShell. In this post, we will explore how to do this for your entire organization via PowerShell to Office365.
NOTE: I have not tested the scripts as I do not have enough mailboxes in my O365 tenant along with not using a ‘.’ in my alias. If the scripts are incorrect, please inform me with the correction and I will update accordingly.
Please make sure that your photos are reviewed before posting, and try to keep the file size of the photos to a minimum. In Office 365, there exists a limitation for the user photo not to be more than 10 KB in size, but I will show you how to get around that limitation.
Having a user photo for each of your users is very beneficial as it personalizes each account to a face in the company. The user photos can be viewed in below locations:
Outlook Web Access
Contact Card
Thumbnail in emails
Outlook Client
Yammer
Lync Client
SharePoint (People Search / Newsfeed)
Steps to take:
Remove the 10KB photo size limitation in Exchange Online
Prepare a folder with all users photos
Update the profile photos via a PowerShell cmdlet.
Connect to Exchange Online with the RPS Proxy Method to remove the 10K size limitation
Connect to Exchange Online PowerShell removing the 10K limitation
NOTE: In the PowerShell cmdlet above, we connected using a different proxy method. This was to overwrite the limitation of uploading the images with size more than 10KB. Using the different proxy method (/?proxyMethod=RPS ) to connect to Office 365 in the above cmdlet accomplishes this.
Prepare a folder locally and place all the photos in that folder
Create a folder named C:/UserPics and make the filename of each photo be the username of that particular user. (i.e. llingerfelt.png) The below script should be able account for aliases that have a ‘.’ in the id as well. (i.e. lance.lingerfelt)
NOTE: From my research, there is no set photo type that is required for the photo. My suggestion would be to keep the photos .png for size constraints while maintaining picture clarity.
Update the profile pictures via PowerShell
Create the following script and name it Photos-Update.ps1
Run Photos-Update.ps1 and the script should upload the photos to Office 365 and apply each photo to the corresponding user.
NOTE: If you’re still having some issues with the alias having a ‘.’ in the name, you can also configure the Photos-Update.ps1 script in this manner to get that working properly:
I wanted to update my picture within my Outlook profile and AD account really quickly without having to go through OWA to do so. I found this cmdlet that will allow for that picture to be changed very quickly via Exchange PowerShell.
NOTE: This can be done with On-Premises Exchange and Exchange Online PowerShell
Old picture within my account
First, download the picture you want to use to the computer that you want to run the cmdlet from. Also, make sure the picture is cropped and centered prior to running the cmdlet. I saved the pic to C:\temp for my scenario. The best format to use would be jpg. I named the file User1_Profile.jpg
Next, open Exchange PowerShell on the computer you saved the pic to and run the following cmdlet to change the photo:
Wanted to do a quick post as I was working on my Hybrid Exchange Environment. I was unable to get the HCW to download and start from the Exchange Control Panel with the link provided on the page. This has happened to me for a while, so I went online and found a link that would work that could be downloaded and reused to open the HCW:
I get a lot of these incidents in my queue after a user has been migrated to O365. For whatever reason, most likely due to the mailbox being moved itself, whether it is the user’s mailbox, the shared mailbox, or both, the connections to the shared mailboxes stop working in Outlook and the user cannot connect to the shared mailbox.
Here is a quick and easy solution to use to disconnect and reconnect the shared mailbox(es) that you lose connectivity to when migrated. This is usually performed on Outlook 2016 and above as most users upgrade their client software when moved to O365.
First, we remove the existing shared mailbox connection:
Click the File > Account Settings > Account Settings.
Select your company email address in the account list.
Click Change > More Settings > Advanced tab > Select the Shared Mailbox > Remove
Click Apply > OK > Next > Finish.
The shared mailbox will now automatically be removed in your Folder pane in Outlook.
Second, we re-add the shared mailbox connection to Outlook:
Click the File > Account Settings > Account Settings.
Select your company email address in the account list.
Click Change > More Settings > Advanced tab > Add
Type the name of the shared mailbox in the window and click OK.
Click Apply > OK > Next > Finish.
The shared mailbox will now automatically be added to your Folder List pane within Outlook.
Note: The above procedure must be followed in order to properly reconnect the shared mailbox. You cannot remove and re-add the mailbox in the same process as that will not reset the connection properly. You must save the settings when disconnecting.
I hope that this will assist everyone when troubleshooting Outlook connectivity issues to shared mailboxes after a migration.
As many of you have knowledge, I am studying for my MS-202 Exam. And, part of the knowledge needed is to be able to migrate mailboxes between on premises and Exchange Online through PowerShell. Here are the steps for the scenario to move a mailbox from on premises to O365:
1. Connect to Exchange Online via PowerShell
If you have read my previous post: Connect to All PowerShell Modules in O365 with one script You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.
2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.
1
$OnPremCred=Get-Credential-Message"Enter the On Premises Exchange Migration Admin account creds"
3. Move a single mailbox.
In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved. Now we initiate the move with the cmdlet. Similar to what you would do in the GUI, this simple mailbox move cmdlet initiates the move request. It has most of the same parameters as a local move request including BadItemLimit, LargeItemLimit, AcceptLargeDataLoss, etc…
Use the following LINK for documentation on the New-MoveRequest cmdlet.
Move a single on premises mailbox to O365
1
2
3
4
5
6
7
8
#Set the identity of the mailbox being moved
$user=Read-Host"Enter the alias or E-Mail Address of the user being migrated: "
#Next Run the cmdlet for the New Mailbox Move Request
New-MoveRequest-Identity$user-remote-RemoteHostName(hybridURL)-TargetDeliveryDomain companyname.mail.onmicrosoft.com-RemoteCredential$OnPremCred-AcceptLargeDataLoss-BadItemLimit500-LargeItemLimit100-BatchName"$user Mailbox Move to O365"
Now with all migration projects, we expect to have to move multiple mailboxes in a single batch. The following will show the process for moving mailboxes in bulk from on premises to O365:
1. Connect to Exchange Online via PowerShell
If you have read my previous post: Connect to All PowerShell Modules in O365 with one script You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.
2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.
1
$OnPremCred=Get-Credential-Message"Enter the On Premises Exchange Migration Admin account creds"
3. Move multiple mailboxes in a single batch.
In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved.
This time you want to create a CSV file using the alias or emailaddress as your header and then list the appropriate value for all the users in your batch group. Save the file locally as MigrationBatch01.csv or a name of your choice.
Use EMailAddress OR Alias as the header
Next you initiate the mailbox moves. When specifying the mailbox identity in the cmdlet, use the respective header in your variable declaration (either $user.EMailAddress OR $user.Alias)
Use the following LINK for documentation on the New-MoveRequest cmdlet.
Move multiple on premises mailboxes to O365 in a Batch
Let’s say you’re an admin that needs to connect to Office365 via PowerShell often. Now, there are many different websites or blogs that will show you how to connect to each session via PowerShell. That can cause a headache since you can end up having five different PowerShell sessions running in five different windows. You end up having to enter a username and password all those times, which can become time consuming.
I want to show you here how to combine all those sessions into one script where, if you’re security is tight enough on your computer, you don’t even have to enter credentials. This way, you can click on one icon and pull up all the O365 PowerShell commands that you’ll need to manage your organization.
First you need to download the following PowerShell Module Installation Files so that your PowerShell Database will have the correct modules installed:
Next, we want to setup the CLI (Command Line Interface) to be too cool for school. I have learned it helps to have knowledge of how to customize the CLI window. You can do all of this in PowerShell ISE or Notepad, which ever you prefer. Here are the commands for the script that I use to setup the CLI:
Setup the CLI for PowerShell
1
2
3
4
5
6
7
8
9
#Set the PowerShell CLI. Make sure you have a directory named "C:\PowerShell"
set-locationc:\PowerShell
$a=(Get-Host).UI.RawUI
$a.BackgroundColor="black"
$a.ForegroundColor="yellow"
$a.WindowTitle="(Your Company Name) PowerShell for ALL O365 PowerShell Services"
$curUser=(Get-ChildItem Env:\USERNAME).Value
functionprompt{"(Your Company Name) O365 PS: $(get-date -f "hh:mm:ss tt")>"}
Next, you want to set your Execution Policy and put in your credentials so that you won’t be prompted to enter the user credentials when you run the script.
NOTE: MAKE SURE YOU KEEP YOUR SCRIPT SAFE AS THE CREDENTIALS ARE VISIBLE WITHIN THE SCRIPT IN PLAIN TEXT!
You can, alternatively, set your script to prompt for credentials every time by using the following:
$LiveCred = Get-Credential
Here is that part of the script:
Setup Execution Policy and Credentials
1
2
3
4
5
6
#Setup Execution Policy and Credentials using your Tenant Admin Credentials
Connect to the Security & Compliance PowerShell: NOTE – This one I still get “Access Denied” when trying to connect. I have looked for an answer to that issue, but have not found one. Please comment with a link if you have an answer so that I can update this script!
Connect to the Compliance and Security Center through PowerShell
1
2
3
4
#Connect the Security & Compliance Center PowerShell Module
Write-Host"Connecting to O365 Security & Compliance Online through PowerShell"-ForegroundColor Green
Write-Host"Be sure to check for any connectivity errors!"-ForegroundColor Green
Write-Host"Also, Remember to run 'Get-PSSession | Remove-PSSession' before closing your PowerShell Window!"-ForegroundColor Green
Write-Host"Successfully connected to all O365 PowerShell Services for (CompanyName)!"-ForegroundColor Green
Now you can create your icon for your desktop so that you can easily access the script. I would save the script to your Scripts directory.
That will usually be C:\Users\’username’\Documents\WindowsPowerShell\Scripts or wherever directory you choose.
To start, right click the desktop and choose New > Shortcut In the Target Field, enter the following for your PowerShell Shortcut, pointing to the path of your script:
Click on the Advanced button and check the box: Run As Administrator Under the General Tab, name your shortcut: (CompanyName) O365 All PowerShell Click OK to save the shortcut to your desktop.
LAST BUT NOT LEAST, RUN THE FOLLOWING COMMAND BEFORE EXITING OR CLOSING YOUR POWERSHELL WINDOW. THIS WILL REMOVE ALL THE SESSIONS YOU’VE CONNECTED TO:
Changes have been made to the HCW and the installation since this original post. Please read the following to gain knowledge of the updates to the tool and the installation.
We wanted to let you know that we are releasing what we consider a significant update to Exchange Hybrid Configuration Wizard (HCW). Along with a handful of small bug fixes, there are four major changes coming that we wanted to share with you:
HCW will no longer enable Federation Trust by default for all installations. Instead, it will only enable Federation Trust if there are Exchange 2010 servers on premises. HCW will call Get-ExchangeServer and if no Exchange 2010 servers are reported, the workflow to enable Federation Trust and subsequently require domain proof will not execute. Note that organization relationships are still created.
When uninstalling the hybrid agent and switching to Classic in the HCW, this action would sometimes fail with a “null reference” error. We have fixed this!
How many of you have hit the HCW 8064 error – unable to configure OAuth, and subsequently had no idea why OAuth failed to configure? Yes, we heard you loud and clear! In this release, we have completely changed the way we enable and configure OAuth. Instead of enabling OAuth at the service layer, we now enable OAuth via a Graph API under the context of the Tenant Admin. This in turn removes the error obfuscation we had with the service layer enablement and allows us to include a detailed error entry in the HCW log. So while you still see the HCW 8064 error in the HCW UI, you can now review the log for the specific error detail which will make it easier to troubleshoot and resolve.
When verifying DNS, we had a fallback mechanism that would reach out to an external site to verify domains. While this fallback mechanism was rarely hit, we received overwhelming feedback to not use this mechanism/site as it was not listed in our IPs & URLs web page. We have removed that fallback and now only use the endpoint “mshybridservice.trafficmanager.net”, which is listed in our endpoints documentation.
Because this is a major version update, the build begins with 17.x vs 16.x. The build number can be found in the top right corner once you download and open the HCW.
Because of the web-based distribution nature HCW uses and this version is a brand new package, you will get all this goodness simply by installing the new HCW from here. The current builds of HCW (16.x) will not automatically update to 17.x build, in fact – you could run the two side-by-side. Once you are on 17.x build – the HCW will then auto-update as usual.
A few additional notes: At this time, we do not anticipate new HCW 16.x builds. Therefore, to continue getting new HCW builds in the future, uninstall the current version of HCW (16.x) and then install the new version (17.x). The new version of HCW has a new dependency, .NET 4.7.2. The installer should take care of this for you, but just so you are aware.
ORIGINAL POST
I’m working on getting certified in Exchange Hybrid Scenarios and Exchange Online configuration as part of my skill set for Exchange. In doing so, I had successfully implemented a complete Full Hybrid Exchange Environment between my Exchange Online Tenant and my On Premises Exchange 2019 Environment last evening.
I wanted to give an update that was posted to my LinkedIn Posting on this. Thank you Brian Day for the vote of confidence and caution that running these cmdlets manually is not supported by Microsoft and that the HCW, like all the Online Microsoft Products, is constantly changing and being updated.
Important Note
As preparation, I bought some Exchange Online Plan 1 licenses which give me a 50 GB mailbox limit and basic mailbox functionality. It does not include the more advanced features such as ATP, or DLP. I am running most of those features through my On Premises Environment. I mainly wanted to be able to place mailboxes in the cloud and have a hybrid setup. My plan was to have mail flow continue through my On Premises environment so that my Exchange Server features would be used and I would not have to change any MX or SPF records. I also had my certificates in place for SSL and OWA so I would want keep mail flow routed that way, through on premises. I do want to be able to have Free/Busy lookups cross-premise so federation would have to be enabled as well. I would also have to enable the MRS proxy on my Exchange Server so that mailbox migration could be implemented cross-premise. I also have previously configured Azure AD Sync along with ADFS for Single Sign On. In my case, another server was not needed as I didn’t have enough mailboxes or real need to split my frontend and backend deployment. Running the Hybrid Configuration Wizard would not open any new ports or change any existing port traffic that was already configured on my firewall. These are just a few of the considerations that need to be looked at when considering a hybrid integration.
So, once I had all those considerations handled in my design, I ran the Hybrid Configuration Wizard. What I want to do in this blog post is to go through the steps that the wizard does in the background to setup the Hybrid Environment as you go through the Wizard.
I mainly used the following blog post as a reference, but have approached it differently by diving into the cmdlets that are run during the process:
1. The HCW validates the On-premises and Online Exchange Connection.
The Hybrid Configuration Wizard checks if it is possible to connect to both servers with PowerShell. It runs the Get-ExchangeServer cmdlet on premises after resolving the server in DNS. It then connects to Exchange Online, authorizing the connection:
3. The HCW collects information on the Exchange online (Office 365) configuration
This task repeats what has been done in the previous step, only for the Exchange online, instead of the on-premises one.
The cmdlets include, in order:
1
2
3
4
5
6
7
Get-OrganizationConfig
Get-OnPremisesOrganization
Get-AcceptedDomain
Get-MigrationEndpoint
4. Federation Trust is determined. If not present, a new Federation Trust and the required certificate will be created on the local Exchange Server
You will be prompted in the Wizard to create a Federation Trust if not present. The following articles explain Federation and its requirements:
Understanding Federation – Link Here Understanding Federated Delegation – Link Here Create a Federation Trust – Link Here
If the activity is finished successfully, a new certificate should appear on the on-premises Exchange Certificates list. The new certificate includes “Federation” in its Subject field. To make sure the certificate is there, you can run a cmdlet: Get-ExchangeCertificate | ft -a -wr
The results will look like this
5. The HCW createsa new Hybrid Configuration Object in the local Active Directory
The HCW will run cmdlets based on the information you provide in the HCW for the certificate, the on premises Exchange Server, the domain(s), and what features you want turned on:
7. The HCW Configures the Organization Relationship between the local server and the cloud.
This configuration is not necessary in minimal hybrid deployment. Since I have a full hybrid deployment configured, the cmdlets were run as needed to configure it. Thanks to the correct configuration, it is possible to synchronize free/busy status of mailboxes and their elements between the on-premises Exchange Environment and Exchange online.
New-OrganizationRelationship-Name'On-premises to O365 - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-TargetApplicationUri'outlook.com'-TargetAutodiscoverEpr'https://autodiscover-s.outlook.com/autodiscover/autodiscover.svc/WSSecurity'-Enabled:$true-DomainNames'LDLNET.mail.onmicrosoft.com'
New-OrganizationRelationship-Name'O365 to On-premises - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-TargetApplicationUri'FYDIBOHF25SPDLT.ldlnet.net'-TargetAutodiscoverEpr'https://autodiscover.ldlnet.net/autodiscover/autodiscover.svc/WSSecurity'-Enabled:$true-DomainNames'ldlnet.net'
Set-OrganizationRelationship-MailboxMoveEnabled:$true-FreeBusyAccessEnabled:$true-FreeBusyAccessLevel LimitedDetails-ArchiveAccessEnabled:$true-MailTipsAccessEnabled:$true-MailTipsAccessLevel All-DeliveryReportEnabled:$true-PhotosEnabled:$true-TargetOwaURL'http://outlook.com/owa/LDLNET.onmicrosoft.com'-Identity'On-premises to O365 - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'
Set-OrganizationRelationship-FreeBusyAccessEnabled:$true-FreeBusyAccessLevel LimitedDetails-MailTipsAccessEnabled:$true-MailTipsAccessLevel All-DeliveryReportEnabled:$true-PhotosEnabled:$true-TargetOwaURL'https://mail.ldlnet.net/owa'-Identity'O365 to On-premises - 48e7bec9-404c-4d24-b59e-4b46b64d7e03'
8. The HCW and setting connectors on both Exchange servers
The HCW checks to see if the connectors are there, if not, it sets them up. During this workflow, four connectors are set – one receive and one send connector for each server. Those connectors guarantee the mail flow between the on-premises and Exchange Online.
New-InboundConnector-Name'Inbound from 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-CloudServicesMailEnabled:$true-ConnectorSource HybridWizard-ConnectorType OnPremises-RequireTLS:$true-SenderDomains''-SenderIPAddresses$null-RestrictDomainsToIPAddresses:$false-TLSSenderCertificateName'.ldlnet.net'-AssociatedAcceptedDomains$null
New-OutboundConnector-Name'Outbound to 48e7bec9-404c-4d24-b59e-4b46b64d7e03'-RecipientDomains'*'-SmartHosts'mail.ldlnet.net'-ConnectorSource HybridWizard-ConnectorType OnPremises-TLSSettings DomainValidation-TLSDomain'mail.ldlnet.net'-CloudServicesMailEnabled:$true-RouteAllMessagesViaOnPremises:$true-UseMxRecord:$false-IsTransportRuleScoped:$false
9. The HCW configures OAuth Authentication across the Hybrid
This LINKexplains how OAuth is configured between Exchange On Premises and Exchange Online. It’s a very good article to read as it shows how to get the Modern Authentication style working. Now the HCW does this for you and at the end of the article, you can run cmdlets to test the validity of the configuration.
Again, look at both of those links to get a little more detail as to what each cmdlet does and how it sets up OAuth. Here are the two cmdlets used to test OAuth:
Run On Premises PowerShell to test OAuth to Exchange Online
In order to be able to move mailboxes between Exchange On Premises and Exchange Online, you have to enable the Exchange Web Services Virtual Directory to use the MRSProxy (Microsoft Replication Service proxy). You also have to set your EWS Virtual Directory to use Basic Authentication. You’ll want to do this before running the HCW or else you will receive the following error when the HCW validates the Migration setup and configuration:
Microsoft.Exchange.Migration.MigrationServerConnectionFailedException: The connection to the server ‘mail.ldlnet.net’ could not be completed. —> Microsoft.Exchange.MailboxReplicationService.RemoteTransientException: The call to ‘https://mail.ldlnet.net/EWS/mrsproxy.svc’ failed. Error details: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. –> The remote server returned an error: (403) Forbidden.. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The remote server returned an error: (403) Forbidden.
Some of the cmdlets run to test Migration and MRS Proxy Settings are as follows:
The HCW runs from final cmdlets to finish up the installation of the Hybrid environment. Here are the cmdlets run:
1
2
3
Get-OnPremisesOrganization
Set-OnPremisesOrganization-Identity'48e7bec9-404c-4d24-b59e-4b46b64d7e03'-Comment'Bunch of letters and numbers'
All this information was found in the setup logs that are in the following directory C:\Users\%username%\AppData\Roaming\Microsoft\Exchange Hybrid Configuration
