Author Lance LingerfeltPosted on

STEPS TO DECOMMISSIONING YOUR EXCHANGE 2010 ON-PREMISES ENVIRONMENT

This was a great article released by the Exchange Team Blog today, and as I have been dealing with MANY customers still having Exchange 2010, I wanted to have this available for quick review! It has great links and steps to consider when finally getting off Exchange 2010.

Best practices when decommissioning Exchange 2010

As many of you know from the previous post regarding Exchange On-Premises Best Practices for Migrations from 2010 to 2016 the end of support for Exchange 2010 is quickly approaching. We’ve created this post to cover the best practices for decommissioning an Exchange 2010 environment after the migration has completed.

Uninstalling Exchange 2010 is as easy as running Setup and selecting to remove the server roles, but there are prerequisites to removing the roles and legacy items left over, which should be removed.

This post is intended to provide best practices to plan for and complete the Exchange 2010 decommission. Please note that since there are many different types of deployments and configurations it is difficult to cover all scenarios, but many of the common steps are included here. Please plan the decommission process carefully.

As a general statement, here are some things that we want to caution against:

  • Do not reuse Exchange 2010 server names (until they have been fully decommissioned).
  • Do not reuse Exchange 2010 server IP addresses (until they have been fully decommissioned).

This post assumes that your organization is maintaining some Exchange presence on-premises, whether Exchange 2013 or 2016 (we do not mention Exchange 2019 in this post because it cannot coexist with Exchange 2010). If your organization has moved all mailboxes to Office 365 and is in a Hybrid environment, we are assuming you will maintain an Exchange footprint per Scenario 2 in How and when to decommission your on-premises Exchange servers in a hybrid deployment.

Preparing for Soft Shut Down

Once you’ve completed the migration from Exchange 2010 to, let’s say, Exchange 2016, you should prepare the 2010 environment prior to decommissioning the servers. The following steps to consider are separated into server roles when preparing for a soft shut down and preparing for the removal of server roles.

Client Access (CAS) Role

Check Server FQDNs

Review all namespaces (e.g. DNS records and load balanced virtual IP addresses) used for client connectivity and ensure they are routing to the 2016 environment. These are all the names that are published for Outlook Anywhere, AutoDiscover, and all Exchange Virtual Directories.

Tip: Verify that all clients such as ActiveSync, Outlook, EWS, OWA, OAB, POP3/IMAP, and Autodiscover are no longer connecting to the legacy Exchange servers. Verification of this can be done by reviewing the servers’ IIS Logs with Log Parser Studio (LPS). LPS is a GUI for Log Parser 2.2 and it greatly reduces the complexity of parsing logs. LPS can parse large sets of logs concurrently (we have tested with total log sizes of >60GB). Please refer to the following blog post with tips and information on using LPS.

Check SCPs

Make sure that the Service Connection Point (SCP) is moved to Exchange 2016 as discussed in the Exchange On-Premises Best Practices for Migrations from 2010 to 2016 post under the Configure Autodiscover SCP for Internal Clients section.

If present, ensure that if the AutoDiscoverServiceInternalURI routes to an Exchange 2016 endpoint. You can also remove this value by setting the AutoDiscoverServiceInternalURI to $Null.

Hub Transport Role

Follow the items below to review all mail flow connectors. We will not be removing connectors themselves, simply auditing to ensure that the server is ready to be decommissioned.

Review the Send Connectors

Review the send connectors and ensure that the legacy servers have been removed and Exchange 2016 servers have been added. Most organizations only permit outbound network traffic on port 25 to a small number of IP addresses, so you may also need to review the outbound network configuration.

Review the Receive Connectors

Review the receive connectors on legacy servers and ensure they are recreated on your Exchange 2016 servers (e.g. SMTP relay; anonymous relay; partner, etc.). Review all namespaces (e.g. DNS records and load balanced virtual IP addresses) used for inbound mail routing and ensure they are terminating against the Exchange 2016 environment. If your legacy Exchange servers have any custom, third-party, or foreign connectors installed (for example, with fax services), ensure that they can be reinstalled on 2016 Exchange servers.

Tip: Check the SMTP logs to see if any outside systems are still sending SMTP traffic to the servers via hard coded names or IP addresses. To enable logging, review Configure Protocol Logging. Also, ensure we have “time coverage” for any apps relaying weekly/monthly emails that may not be caught in a small sample size of SMTP Protocol logs. There is a great script available here that can help find any applications that may be relaying off your legacy environment.

In general, the decommissioning process is a great time to audit your mail flow configuration to ensure that all the connectors are properly configured and secured. Maybe it’s time to get rid of any of those Anonymous Relay connectors that may be in use in your environment. Or, if Hybrid, possibly relay against Office 365.

Transport Rules

Exchange 2010 base transport rules are held in a different AD container than Exchange 2013 and newer rules. When installing Exchange 2016 in your environment it will import those Exchange 2010 based rules. However, any changes to Exchange 2010 rules after a later version of Exchange is installed must also be applied to your Exchange 2016 rules. This is further explained here under section Coexistence with Exchange 2010.

Run the following command to get all your Exchange Transport Rules. Must be run on Exchange 2016 to see all rules.

Compare the rules with RuleVersion of 14.X.X.X to those with 15.1.X.X. If any Exchange 2010 rules don’t exist on Exchange 2016, they must be created. Also review all settings of each Exchange 2010 rule and replicate them to Exchange 2016.  

Mailbox Role

Identity and move all Exchange 2010 mailboxes to Exchange 2016

Decommissioning Exchange 2010 cannot be initiated until all mailboxes have been moved to Exchange 2016. As an example, we cannot decommission Exchange 2010 Hub Transport servers completely until all of the mailboxes are moved off the legacy platform, this is due to how Delivery Groups are handled.

We encourage using the newest Exchange platform to process any move requests. If moving to Exchange 2016, move all mailboxes via Exchange 2016. Also, ensure that once all moves are completed, and that all associated Move Requests are removed as well. Any lingering move requests or mailboxes will prevent uninstallation of Exchange 2010.

To move all user mailboxes, run the following command to identify the mailboxes, and then plan to move them to the new platform.

Tip: Ensure that Archives are included with “Get-Mailbox -Archive” if you used Exchange Archives in 2010. Also, do not forget about your Discovery Search mailboxes – these can be found with: Get-Mailbox -Filter { RecipientTypeDetails -eq “DiscoveryMailbox”}. These will need to be moved (if they haven’t yet already), to Exchange 2016 as well.

Identify and Move Arbitration Mailboxes to Exchange 2016

It’s necessary to move the arbitration mailboxes from Exchange 2010 to 2016 for many Exchange Services to work properly, including the Exchange Admin Center (EAC). This is typically executed when Exchange 2016 is first installed, however, if that was missed, we will ensure that is handled now. The process to move is defined at: Move the Exchange 2010 system mailbox to Exchange 2013+. To verify which system mailboxes are located on 2010, use PowerShell on your Exchange 2010 server with the following example:

Note: If any mailboxes are present, move them to an Exchange 2016 database.

OAB Generation

Installing first Exchange Server 2013+ into Exchange 2010 organization creates a new OAB. It also marks the new OAB as default. The Exchange 2010 OAB is not used by Exchange 2013+ servers so moving the OAB is not necessary. Move the OAB to another Exchange 2010 server, if you are removing an Exchange 2010 server that’s currently hosting the OAB, and there are other Exchange 2010 servers in the org. If you are removing the last Exchange 2010 server in the org, remove the OAB.

Migrate All Legacy Public Folders

Verify that all the public folders have been migrated to Exchange Online, Office 365 Groups, or Exchange Modern public folders.

Mail Enabled Public Folders (MEPF) consideration

If the following is true:

  • Exchange Server 2010 public folders are migrated to Exchange Online
  • Exchange Server 2013/2016 was introduced on-premises
  • MEPF’s are still used on-premises to send emails to Exchange Online

In that case, you may need to run the SetMailPublicFolderExternalAddress.ps1 script to ensure Exchange 2013+ servers can continue sending emails to Exchange Online MEPFs.

Decommission the Database Availability Group (DAG)

Assuming best practices were followed for the Exchange 2010 environment, we will have a DAG for HA/DR capabilities. Now that all mailboxes have been removed from the 2010 environment, we are ready to tear down this DAG to move forward with decommissioning Exchange 2010.

Remove Database Availability Group (DAG) Copies

First, we start with the copies. For every mailbox database copy in the environment hosted on Exchange 2010, we will need to remove the Mailbox Database Copy. This can be done via the UI, or via PowerShell:

NOTE: Removing the copy will not remove the actual .edb database file from the Server.

Remove All Nodes from Database Availability Group(s) (DAG)

For each Exchange 2010 server in the environment, we will need to remove the individual server from the DAG. This is evicting the server from the cluster. This can be done via the UI, or through PowerShell.

Remove DAGs

Lastly, once the Database copies are removed, and the servers are evicted from the cluster, the last thing is to finally remove the DAG from the environment. This can be done with the following PowerShell command:

Tip: If you have an even-membered DAG, and leveraged a File Share Witness, don’t forget to decommission the file share witness that was used for the Exchange 2010 DAG.

Unified Messaging Role

Configuration steps are required to move Exchange 2010 UM to Exchange 2016 servers. The following link can be used to guide through removal of UM from Exchange 2010. If moving to a third-party UM solution, remove the UM components to allow un-installation of the UM role.

Edge Role

If you have an Edge server, you will need to install Exchange 2016 Edge and recreate the Edge Subscription on the E2016 server. This is further documented here.

Other

As mentioned in the beginning of the document, due to so many different types of deployments and configurations, it’s difficult to cover all scenarios however it’s recommended to check any other possible scenarios that apply to your environment.

Third Party Applications

Make a list of applications that may be using Exchange 2010 (e.g. EWS, mail transport, database-aware) and make sure to configure these applications to start using Exchange 2016 infrastructure.

Shut-Down Exchange 2010 Servers

Test shutting down the Exchange servers for a few days to a few weeks to see if there are any issues. You are auditing for any applications that are trying to connect to the Exchange 2010 servers or trying to send email through the Exchange 2010 servers.  Enabling protocol logging on the Hub Transport roles prior to shutting down the servers is an option. That way if any mail is processing through these servers, upon restart, the logging will begin immediately.  If applications or servers are trying to connect you can remediate those or power on the Exchange 2010 servers until remediation can happen.

Tip: Check Active Directory DNS Zone settings to see if DNS Scavenging is enabled.  If this is enabled, the DNS record could become stale during the shutdown time frame and cause DNS issues for the Exchange 2010 server.

Preparing for Removal of Server Roles

As you begin the process of removing servers, you should go through the list below and ensure you have everything tested and ready to go.

CAS

Remove CAS Arrays

Remove Any Exchange 2010 Client Access Arrays from Active Directory and DNS. Refer to the following document to remove the Client Access Array object with Shell using the following example:

Be sure to also remove any references in DNS to the CAS Array Name.

Remove Unused 2010 ASAs

If you followed either the Best practices for Migrations blog or the Coexistence with Kerberos blog, we recommend that any old alternate service accounts (ASAs) used for E2010 be removed. If you are using a different namespace than Exchange 2016, please verify old SPNs are also removed.

Remove Exchange 2010 OAB

Use the following command to remove Exchange 2010 OAB:

Remove Mailbox Databases

Now that all mailboxes are migrated from the Exchange 2010 platform, and the DAG is properly removed, we will want to decommission any leftover databases from the Exchange 2010 environment. To remove all Exchange 2010 databases, review the output of the following, and remove individually:

And then remove the database with:

NOTE: If there are any mailboxes currently residing on the database, we will not let you remove the database, it will fail with the following error:

e2010decom1.jpg
Remove Legacy Public Folders

If you chose not to migrate public folders, refer to the following document to remove public folders with either EMC or Shell using the following example:

Remove Legacy Public Folder Databases

Refer to the following document to remove the public folder databases with PowerShell using the following example:

Tip: Remember the .edb files linger after the above is done. Feel free to delete, backup, or do with these as you please.

Uninstall Exchange 2010

It’s recommended to uninstall in the following order: CAS, Hub, UM (if any), then Mailbox.  

Starting the Uninstall Process

When you begin the uninstall process, close EMC, EMS, and any additional programs that could delay uninstall process (i.e. programs using .NET assemblies; antivirus and backup agents are examples). You can either run Exchange 2010 Setup.exe or navigate to Control Panel to modify or remove Exchange 2010 (either server roles or the entire installation). Specific steps are discussed in Modify or Remove Exchange 2010.

Tip: Exchange will protect itself! If you properly uninstall via Add/Remove Programs, it will ensure that it is ready to be uninstalled via Readiness Checks! If all the above prep work is completed before hand, it should uninstall just fine.

After Uninstall of Exchange 2010

After uninstalling Exchange there will be some general “housekeeping” tasks. These may vary depending on the steps taken during your upgrade and depending on your organization’s operational requirements.

Examples include:

  • Removing the legacy Exchange computer accounts from AD (including the DAG’s Cluster Name Object and any Kerberos ASA object).
  • Removing the legacy Exchange name records from DNS (including the DAG’s Cluster Name Object and any Kerberos ASA object).
  • Ensure the folder on the DAG file share witness (FSW) servers were successfully removed, possibly removing Exchange’s rights on the server if it isn’t serving double duty for Exchange 2016.
  • Removing old load balanced IP addresses and routes from your network load balancer.
  • Remove old firewall rules that open ports to Exchange 2010 environment.
  • Removing and disposing of the legacy Exchange environment’s physical equipment.
  • Deleting of the legacy Exchange environment’s virtual machines.

Conclusion

With the uninstall of the last server, hopefully Exchange 2010 treated your organization well. The Exchange product team takes great pride of the success of the platform and hope that you see the same success with Exchange 2016 (or Exchange Online!). Messaging sure has come a long way since it was released way back in 2009.

REFERENCES
Exchange Team Blog article on Decommissioning Exchange 2010 On-Premises

CHECK FOR CONTINUED UPDATES!
THANKS FOR STOPPING BY!

Author Lance LingerfeltPosted on

Why go to Exchange 2019 from 2016 and 2013?

I get asked this a lot in my travels, “What’s the difference with Exchange 2019 and why should we go to it other than the fact that it is the latest version released. I wanted to post my findings from articles that I found to help explain some of the improvements and differences with running Exchange 2019.

Exchange Server 2019 brings a new set of technologies, features, and services to Exchange Server, the messaging platform that provides email, scheduling, and tools for custom collaboration and messaging service applications.

What is NOT in Exchange 2019…

There are some things that have been discontinued in Exchange 2019 which make the decision to go to it important for some companies.

Architecture

FeatureComments and mitigation
Unified Messaging (UM)Unified Messaging has been removed from Exchange 2019. We recommend that Exchange 2019 organizations transition to Skype for Business Cloud Voice Mail.

This is a big deal for some companies as they rely on Unified Messaging to handle their Voice Messaging. There are some articles available to view to assist in transitioning from UM to the new voicemail features with O365. I will post them as I find them in this blog. Keep tuned in for updates!

Here are some other deprecated features:

Client Access server roleThe Client Access server role has been replaced by Client Access services that run on the Mailbox server role. The Mailbox server role now performs all functionality that was previously included with the Client Access server role. For more information about the new Mailbox server role, see Exchange Server architecture.
MAPI/CDO libraryThe MAPI/CDO library has been replaced by Exchange Web Services (EWS), Exchange ActiveSync (EAS), and Representational State Transfer (REST)* APIs. If an application uses the MAPI/CDO library, it needs to move to EWS, EAS, or the REST APIs to communicate with Exchange 2019.

De-emphasized Features

The following features are being de-emphasized in Exchange 2019 and may not be included in future versions of Exchange.

  • Third-party replication APIs
  • RPC over HTTP
  • Database availability group (DAG) support for failover cluster administrative access points (You can have IPLess DAGs now)

What’s new when upgrading to Exchange 2019?

Security

  • Windows Server Core support: Running Exchange on a Windows deployment with less surface area means less attack surface area and fewer components to service.
  • Block external access to Exchange admin center (EAC) and the Exchange Management Shell: You can use Client Access Rules to only allow administration of Exchange from the internal network instead of using complex network and firewall rules.
  • TLS 1.2 is the only version that’s enabled by default: Exchange Server 2019 includes important changes to improve the security of client and server connections. The default configuration for encryption will enable TLS 1.2 only and disable support for older algorithms (namely, DES, 3DES, RC2, RC4 and MD5). It will also configure elliptic curve key exchange algorithms with priority over non-elliptic curve algorithms. In Exchange Server 2016 and later, all cryptography settings are inherited from the configuration specified in the operating system.

Performance

  • Improved search infrastructure: The completely rebuilt search infrastructure for cloud scale and reliability in Exchange Online is now available in Exchange 2019. This new search infrastructure allows for indexing of bigger files, simpler management, and better search performance.
  • Faster, more reliable failovers: The changes to the search architecture result in significantly faster and more reliable failover over between servers.
  • Metacache database: Improvements at the core of Exchange’s database engine enable better overall performance and take advantage of the latest storage hardware, including larger disks and SSDs.
  • Modern hardware support: Exchange now supports up to 256 GB of memory and 48 CPU cores.
  • Dynamic database cache: The information store process employs dynamic memory cache allocation optimizing memory usage to active database usage.

Clients

  • Calendar – Do Not Forward: This is similar to Information Rights Management (IRM) for calendar items without the IRM deployment requirements. Attendees can’t forward the invitation to other people, and only the organizer can invite additional attendees.
  • Calendar – Better Out of Office: Additional options when you won’t be in the office. Key options include: add an event to your calendar that shows you as Away/Out of Office, and a quick option to cancel/decline meetings that will happen while you’re away.
  • Calendar – Remove-CalendarEvents cmdlet: Enables administrators to cancel meetings that were organized by a user that has left the company. Previously, conference rooms or meeting attendees would have these defunct meetings permanently on their calendars.
  • Assign delegate permission via PowerShell: Updates to the Add-FolderPermissions cmdlet so administrators can assign delegate permissions.
  • Email address internationalization (EAI): Email addresses that contain non-English characters can now be routed and delivered natively.

Exchange 2019 architecture

Today, CPU horsepower is significantly less expensive and is no longer a constraining factor. With that constraint lifted, the primary design goal for Exchange 2019 is for simplicity of scale, hardware utilization, and failure isolation. With Exchange 2019, we reduced the number of server roles to two: the Mailbox and Edge Transport server roles.

Unified Messaging (UM) has been removed from Exchange 2019. Other than that, the Mailbox server in Exchange 2019 includes all of the server components from the Exchange 2013 Mailbox and Client Access server roles:

  • Client Access services provide authentication, limited redirection, and proxy services. Client Access services don’t do any data rendering and offer all the usual client access protocols: HTTP, POP and IMAP, and SMTP.
  • Mailbox services include all the traditional server components found in the Exchange 2013 Mailbox server role except Unified Messaging: the backend client access protocols, Transport service, and Mailbox databases. The Mailbox server handles all activity for the active mailboxes on that server.

The Edge Transport role is typically deployed in your perimeter network, outside your internal Active Directory forest, and is designed to minimize the attack surface of your Exchange deployment. By handling all Internet-facing mail flow, it also adds additional layers of message protection and security against viruses and spam, and can apply mail flow rules (also known as transport rules) to control message flow.

For more information about the Exchange 2019 architecture, see Exchange architecture.

Along with the new Mailbox role, Exchange 2019 now allows you to proxy traffic from Exchange 2013 Client Access servers to Exchange 2019 mailboxes. This new flexibility gives you more control in how you move to Exchange 2019 without having to worry about deploying enough front-end capacity to service new Exchange 2019 servers.

MAPI over HTTP

MAPI over HTTP is now the default protocol that Outlook uses to communicate with Exchange. MAPI over HTTP improves the reliability and stability of the Outlook and Exchange connections by moving the transport layer to the industry-standard HTTP model. This allows a higher level of visibility of transport errors and enhanced recoverability. Additional functionality includes support for an explicit pause-and-resume function, which enables supported clients to change networks or resume from hibernation while maintaining the same server context.

Note: MAPI over HTTP isn’t enabled in organizations where the following conditions are both true:

  • You’re installing Exchange 2019 in an organization that already has Exchange 2013 servers installed.
  • MAPI over HTTP wasn’t enabled in Exchange 2013.

While MAPI over HTTP is now the default communication protocol between Outlook and Exchange, clients that don’t support it will fall back to Outlook Anywhere (RPC over HTTP).

Outlook on the Web
(formerly known as Outlook Web App)

Outlook Web App is now known as Outlook on the web, which continues to let users access their Exchange mailbox from almost any web browser.

NOTE: Supported Web browsers for Outlook on the web in Exchange 2019 are Microsoft Edge, Internet Explorer 11, and the most recent versions of Mozilla Firefox, Google Chrome, and Apple Safari.

The former Outlook Web App user interface has been updated and optimized for tablets and smart phones, in addition to desktop and laptop computers. New Exchange 2019 features include:

  • Platform-specific experiences for phones for both iOS and Android.
  • Premium Android experience using Chrome on devices running Android version 4.2 or later.
  • Email improvements, including a new single-line view of the Inbox with an optimized reading pane, archiving, emojis, and the ability to undo mailbox actions like deleting a message or moving a message.
  • Contact linking the ability for users to add contacts from their LinkedIn accounts.
  • Calendar has an updated look and new features, including email reminders for Calendar events, ability to propose a new time in meeting invitations, improved search, and birthday calendars.
  • Search suggestions and refiners for an improved search experience that helps users find the information they want, faster. Search suggestions try to anticipate what the user’s looking for and returns results that might be what the user is looking for. Search refiners will help a user more easily find the information they’re looking for by providing contextually-aware filters. Filters might include date ranges, related senders, and so on.
  • New themes Thirteen new themes with graphic designs.
  • Options for individual mailboxes have been overhauled.
  • Link preview which enables users to paste a link into messages, and Outlook on the web automatically generates a rich preview to give recipients a peek into the contents of the destination. This works with video links as well.
  • Inline video player saves the user time by keeping them in the context of their conversations. An inline preview of a video automatically appears after inserting a video URL.
  • Pins and Flags which allow users to keep essential emails at the top of their inbox (Pins) and mark others for follow-up (Flags). Pins are now folder specific, great for anyone who uses folders to organize their email. Quickly find and manage flagged items with inbox filters or the new Task module, accessible from the app launcher.
  • Performance improvements in a number of areas across Outlook on the web, including creating calendar events, composing, loading messages in the reading pane, popouts, search, startup, and switching folders.
  • New Outlook on the web action pane that allows you to quickly click those actions you most commonly use such as New, Reply all, and Delete. A few new actions have been added as well including Archive, Sweep, and Undo.

Document collaboration
(On-Premises and in O365)

Exchange 2019, along with SharePoint Server 2019 and SharePoint Online, enables Outlook on the web users to link to and share documents that are stored in OneDrive for Business in an on-premises SharePoint server instead of attaching files to messages. Users in an on-premises environment can collaborate on files in the same manner that’s used in Office 365.

When an Exchange 2019 user receives a Word, Excel, or PowerPoint file in an email attachment, and the file is stored in OneDrive for Business or on-premises SharePoint, the user will now have the option of viewing and editing that file in Outlook on the web alongside the message. To do this, you’ll need a separate computer in your on-premises organization that’s running Office Online Server.

Exchange 2019 also brings the following improvements to document collaboration:

  • Saving files to OneDrive for Business.
  • Uploading a file to OneDrive for Business.
  • Most Recently Used lists populated with both local and online files.

Office 365 hybrid and the HCW

The Hybrid Configuration Wizard (HCW) that was included with Exchange 2013 is moving to become a cloud-based application. When you choose to configure a hybrid deployment in Exchange 2019, you’ll be prompted to download and install the wizard as a small app. The wizard will function the same in previous versions of Exchange, with a few new benefits:

  • The wizard can be updated quickly to support changes in the Office 365 service.
  • The wizard can be updated to account for issues detected when customers try to configure a hybrid deployment.
  • Improved troubleshooting and diagnostics to help you resolve issues that you run into when running the wizard.
  • The same wizard will be used by everyone configuring a hybrid deployment who’s running Exchange 2013 or later.

In addition to Hybrid Configuration Wizard improvements, multi-forest hybrid deployments are being simplified with Azure Active Directory Connect (AADConnect). AADConnect introduces management agents that will make it significantly easier to synchronize multiple on-premises Active Directory forests with a single Office 365 tenant.

Exchange ActiveSync clients will be seamlessly redirected to Office 365 when a user’s mailbox is moved to Exchange Online. To support this, ActiveSync clients need to support HTTP 451 redirect. When a client is redirected, the profile on the device is updated with the URL of the Exchange Online service. This means the client will no longer attempt to contact the on-premises Exchange server when trying to find the mailbox.

Secure Messaging, Policy, and Compliance

Data loss prevention

To comply with business standards and industry regulations, organizations need to protect sensitive information and prevent its inadvertent disclosure. Examples of sensitive information that you might want to prevent from leaking outside your organization include credit card numbers, social security numbers, health records, or other personally identifiable information (PII). With a DLP policy and mail flow rules (also known as transport rules) in Exchange 2019, you can now identify, monitor, and protect 80 different types of sensitive information with new conditions and actions:

  • With the new condition Any attachment has these properties, including any of these words, a mail flow rule can match messages where the specified property of the attached Office document contains specified words. This condition makes it easy to integrate your Exchange mail flow rules and DLP policies with SharePoint, Windows Server 2012 R2 File Classification Infrastructure (FCI), or a third-party classification system.
  • With the new action Notify the recipient with a message, a mail flow rule can send a notification to the recipient with the text you specify. For example, you can inform the recipient that the message was rejected by a mail flow rule, or that it was marked as spam and will be delivered to their Junk Email folder.
  • The action Generate incident report and send it to has been updated to enable the notification of multiple recipients by allowing a group address to be configured as the recipient.

In-place Archiving, retention, and eDiscovery

Exchange 2019 includes the following improvements to In-Place Archiving, retention, and eDiscovery to help your organization meet its compliance needs:

  • Public folder support for In-Place eDiscovery and In-Place Hold: Exchange 2019 integrates public folders into the In-Place eDiscovery and Hold workflow. You can use In-Place eDiscovery to search public folders in your organization, and you can put an In-Place Hold on public folders. And similar to placing a mailbox on hold, you can place a query-based and a time-based hold on public folders. Currently, you can only search and place a hold on all public folders. In later releases, you’ll be able to choose specific public folders to search and place on hold. For more information, see Search and place a hold on public folders using In-Place eDiscovery.
  • Compliance Search: Compliance Search is a new eDiscovery search tool in Exchange 2019 with new and improved scaling and performance capabilities. You can use it to search very large numbers of mailboxes in a single search. In fact, there’s no limit on the number of mailboxes that can be included in a single search, so you can search all mailboxes in your organization at once. There’s also no limit on the number of searches that can run at the same time. For In-Place eDiscovery in Exchange 2019, the limits are the same as in Exchange 2013: you can search up to 10,000 mailboxes in a single search and your organization can run a maximum of two In-Place eDiscovery searches at the same time.

Indexing and Search Architecture

In Exchange 2019, the search architecture has been redesigned. It is now based on the same engine as the modern search engines are and is directly on the mailbox in Exchange 2019. There is no content index database attached to the mailbox database as in previous versions of Exchange Server. Previously, search was a synchronous operation that was not very fault-tolerant. The new architecture is asynchronous and decentralized. It distributes the work across multiple servers and keeps retrying if any servers are too busy. This means that we can return results more reliability, and faster.

Another advantage of the new architecture is that search scalability is improved. The number of mailboxes you can search at once using the console has increased from 5k to 10k for both mailboxes and archive mailboxes, allowing you to search a total of 20k mailboxes at the same time.

ENJOY YOUR UPGRADE!
LEARN, DO, LIVE!

REFERENCES:
What is new in Exchange Server
What is discontinued in Exchange Server
Exchange Server TLS Guidance
Exchange Architecture

Author Lance LingerfeltPosted on

Welcome to my IT Blog…

This blog is going to be a dedicated repository linked to pages, posts, documentation, links, and information that I find during my troubleshooting processes within the IT world. Some posts will contain code snippets that you can use with your own work if needed. Feel free to comment and get this thing rolling!

I now have contributors! Please welcome their input as valued colleagues in the IT Industry! Be on the lookout for their blog posts and please comment!

LDLNET LLC – Your source for Professional IT Services
Author Lance LingerfeltPosted on

Using RDP to access an Azure AD Domain Joined Computer

I have a VM that is joined to my Azure AD test tenant domain. I was having issues using RDP to access the box with my Azure AD credentials (username@tenant.onmicrosoft.com). I kept getting the following when trying to connect:

Azure AD Credentials would not work to RDP to the client.

So I started researching and found that this was an common issue that many have started to face with their Azure AD Joined machines. Unfortunately, at this time it isn’t quite as easy as “open up a new RDP connection, type in the computer, type my email, and connect”.  Here are the steps to connect a session to that Azure AD joined computer.

Steps to connect RDP to an Azure AD joined computer.

First, open remote desktop as if you were going to connect to any other computer. Type in the computer name or IP address and expand the the Show Options section. Next, click the Save As button to save the RDP file to your computer. At this point you can close the Remote Desktop Connection window as it isn’t needed any longer.

Next, open Notepad. Click File -> Open -> location your RDP file that was saved in the previous step. 

Go to the very bottom of the list of parameters and add the following two lines:

enablecredsspsupport:i:0
authentication level:i:2

Save the changes to the .rdp file

NOTE: You can also add your username that will be used to connect to the session in the file as well:

username:s:.\AzureAD\YOURusername@YOURtenantname.onmicrosoft.com

Example RDP File

Now you are ready to connect! Double click on the RDP file and connect to the Azure AD Joined computer.

KEEP RESEARCHING!
STAY POSITIVE! THE WORLD WILL CHANGE FOR THE BETTER FOR ALL OF US!

REFERENCES:
Remote Desktop to Azure AD Joined Computer

Author Lance LingerfeltPosted on

Connect an MCT Credited Azure Subscription to separate Azure Tenant

As many of you know I have an existing Azure/M365 Tenant that I use with my company as many of you all do as well. When you get an MCT Certification, you get access to a monthly credit in Azure.

So I clicked on Activate and when activating the subscription, it created a new Azure tenant that was linked to my MCP ID and not my LDLNET ID. The problem here was that my Microsoft Identity was a different email address (@live.com) from my tenant Microsoft Identity (@ldlnet.net).

So now I have a new tenant that I really don’t need, so I started thinking, could I transfer the subscription to my LDLNET tenant and keep the monthly credit. The answer is Yes AND No.

Here is what I had to do to transfer the subscription to my LDLNET Directory

First, I created a guest account in my LDLNET tenant for the @live.com email address and then temporarily gave the account Global Admin privileges. This was so that I could access the subscription when transferred and assure that the proper accounts that needed subsequent access to the subscription get what the owner permissions by logging on with the @live.com account in the LDLNET tenant. I then activate the account in LDLNET.

NOTE: This is probably NOT the most secure option to start, but I will update as I find the article’s that define least privilege for setting this up. I’ve seen a couple of articles, but it wasn’t the exact same way. The thing here is that the billing cannot be transferred since it is being handled by Microsoft Directly with the credits. So, I have to keep the @live.com account active in the LDLNET tenant so that it bills correctly.

Next, I go to the setup tenant and look at the subscription Overview:

At the top of the screen, I choose Change Directory. Since my @Live.com account was an admin for the LDLNET Directory now, I could choose the directory on the following screen:

Change Directory to the destination tenant.

NOTE: I couldn’t change the billing on the setup tenant nor transfer it since it was through Microsoft, but why would I want to anyway since it’s my credit that was given to me monthly. Also, on my visual studio subscription, I made sure that my @ldlnet.net address was an alternate access account on the subscription. I want to make sure the credit stays after this month!!

So, I received the email asking to accept the transfer and clicked Accept The Transfer:

Accept The Transfer E-Mail

Once the subscription was accepted and transferred to LDLNET, I logged into LDLNET tenant with my @Live.com account. I then went to the subscription and made sure to add all necessary accounts to the subscription so that they would get access:

Once completed, I logged in with my Original Global Admin account and changed the @Live.com account permissions to a Global Reader in my LDLNET tenant, then gave them Owner Access permissions to the Subscription specifically.

And that has completed the transfer. Hopefully, the subscription will continue with the monthly credit as per my MCT Certification allows. I will update if something changes. If you have a better way to do this, please comment and I will be happy to verify it and post!

KEEP THE COMMENTS COMING!
THANKS FOR READING!

Author Lance LingerfeltPosted on

New Certification Achieved Today

I have been working on updating my skillset to M365 and passed the final exam today to achieve it. I am hoping that the certification will assist me with attaining work moving forward.

You can verify at the following URL:
https://www.youracclaim.com/badges/8bb7d636-b898-43ec-b283-0dea03586896/public_url

MAINTAIN POSITIVE ATTITUDE!
SUCCESS WILL ARRIVE IN DUE TIME!

Author Lance LingerfeltPosted on

Remote Desktop Licensing Mode is Not Configured when configuring Remote Desktop Services

I recently setup a backend RDP server so that I could test our remote services. I went through the installation process and installed my RDP license on the server successfully. The problem was that I was not getting an error when logging onto the server locally to check the status of the RDP Server:

remote desktop licensing mode is not configured
Error Example for RDP Licensing

I also had Events within Event Viewer:

Log Name: System
Source: Microsoft-Windows-TerminalServices-Licensing
Date: 6/24/2020 3:44:16 PM
Event ID: 18
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: SRV2016-02.ldlnet.net
Description:
The Remote Desktop license server “SRV2016-02” has not been activated and therefore will only issue temporary licenses. To issue permanent licenses, the Remote Desktop license server must be activated.

Usually, this error appears as a notification popup in the bottom right-hand corner of the screen saying:

Remote Desktop licensing mode is not configured
Remote Desktop Services will stop working in xx days. On the RD Connection Broker server, use Server Manager to specify the Remote Desktop licensing mode and the license server.

And when you click on this notification popup, it doesn’t redirect you anywhere and it gets simply disappeared which is a quite frustrating situation.

I thought I had this set properly, but the RDP Licensing Diagnoser application told me that I needed to choose the licensing configuration to distribute for Per Device OR Per User. The licenses I installed were per device so I did some research.

I found out that this had to be configured via the Registry, PowerShell, or could be configured through GPO. I chose to do GPO since that would always apply on my servers in my domain.

How To configure the Remote Desktop Licensing Mode through the Registry

Here’s how to change the licensing mode for Remote Desktop session host using the registry editor and get rid of the error message Remote Desktop Services will stop working in xx days:

At first, press Windows + R keys together and then type regedit in the Run dialog box and press Enter key.

regedit windows 10

Next, in the left pane of the Registry Editor, navigate to the following registry key:

licensing mode for the remote desktop session host is not configured

Next in the right pane, double-click on the LicensingMode to edit its value and then change the Value data according to your requirement:

Set the Value data 2 for Per Device RDS licensing mode
Set the Value data 4 for Per User RDS licensing mode

remote desktop services will stop working

Finally, click on the OK button to save the changes.

Now, restart your computer and check if the Remote Desktop licensing mode is not configured issue on Windows Server has been resolved or not.

Once you changed the licensing mode, now everything will be reported accurately and the Remote Desktop session host will recognize the licensing configuration.

How To configure the Remote Desktop Licensing Mode through a Group Policy Object (GPO)

First Logon to a machines that has Group Policy Tools Installed, press Windows + R keys together, type gpedit.msc, and press Enter key.

gpedit-windows-10

Within Group Policy Editor, create a new GPO and link it to the level that you need to link it to. (In my case, the domain level.) Navigate to:

Computer Configuration > Policies > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Licensing.

change remote desktop licensing mode

Next, double-click on the “Use the specified Remote Desktop license servers” setting and then select Enabled option. Finally, enter the names of the license servers (host name or IP address) and then click on the OK button.

use the specified remote desktop license servers

Similarly, double-click on the “Set the Remote Desktop licensing mode” setting and then select Enabled option. Finally, set the licensing mode (Per Device or Per User) and then click on the OK button.

set the remote desktop licensing mode

Once all these changes are done, close Group Policy Editor, go to the RDP Licensing Server and run a gpupdate /force to refresh Group Policy.

Now, when you open the Remote Desktop Licensing Diagnoser, you shouldn’t see any errors like the remote desktop licensing mode is not configured on windows server or any kind of issues regarding your licenses.

Corrected Licensing Diagnoser Result

How To configure Remote Desktop Licensing Mode through PowerShell

You can also use PowerShell to set the Licensing Mode via the Set-RDLicenseConfiguration cmdlet from the RemoteDesktop PowerShell Module which is installed with Remote Desktop Services:

KEEP POSITIVITY ON YOUR SIDE!
CONTINUE TO LEARN!

REFERENCES:
How to Fix Remote Desktop Licensing Mode is Not Configured
Set-RDLicenseConfiguration

Author Lance LingerfeltPosted on

Grant an External User Guest Access to your M365 Tenant

Microsoft365 allows the tenant administrators to grant external users access to content in their tenant by setting them up as a guest in their M365 Tenant. Microsoft365 provides a guest access feature that you can use to grant content access to contractors, partners or others who need access to certain content.

However, the process of setting up a guest user works differently from that of setting up a normal, licensed user from within your organization.

By default, Microsoft365 Admin Center contains a Guest Users screen. You will also notice, however, that this screen does not contain an option to create a guest user. In fact, the only things that you can do are search for a user or delete a user.

Limited Access to Administrate Guest Users in M365

Being that the Guest Users screen doesn’t give you a way to create a guest user, you will need to either delve into PowerShell or perform the task within Azure Active Directory. I prefer using PowerShell, and will write a post about how to perform this via PowerShell, but unless you need to create a large number of guest users, it is usually going to be easier to use the GUI. Below is how to create a guest user via Azure AD.

To create a guest user, expand the Admin Centers container and then click on Azure Active Directory. When the Azure Active Directory Admin Center opens, click on the Users container. You can see that just to the right of the New User option, there is an option to create a New Guest User.

Create New Guest User

NOTE: Creating a guest user account isn’t like creating a normal user account. Rather than providing the account details and clicking a Create button, you will instead need to send an invitation to the user.

Make Sure You Verify Their E-Mail Address Beforehand!!!

Choose Invite User > Enter the Identity Information

Initial Data Entry

Next Enter A Personal Message (optional) > Choose their Group Membership > Update any AAD or M365 Permissions under Roles > Update their Sign In Settings > Click Invite to send the invitation

Enter Data and Settings Then Click Invite Button

After a few minutes, the specified user will receive an e-mail invitation that looks something like the one shown below. The recipient will need to click the Accept Invitation button and accept the terms of use.

Example of Email Generated Invitation

When the guest user completes the registration process, they are logged into Microsoft365 however, there are no applications initially available to the user. This is because unlike a standard user, external users do not automatically get access to applications.

User Has Verified Access and Accepted the Invitation

If you go back to the Guest Users screen, you will see the newly created guest user listed (you may have to refresh the screen). As previously noted, you can’t do much from this screen. You can, however, click on the user to see a few extra details now. Example is below.

More Details Available

The way that you grant an external user access to data is to add the user to a group that has access to the data. Let’s suppose, for example, that for whatever reason, you need to add an external user to a Teams Group named Microsoft Exchange Guys. To do so, you would go to the Groups folder within the Microsoft 365 Admin Center, click on the Microsoft Exchange Guys group, and then edit the Membership list, as shown below.

After clicking the Edit button, click on Add Members and then select the external user that you wish to add. Click Save to complete the process,

The New Guest User Will Show When Searching To Add Users To The Group

If you now go back to the Group’s membership, you are able to see the Microsoft Exchange Guys group membership showing the new guest user as a member.

Guest User Has Been Added To The Group

Granting access in this way does not provide the external user with blanket access to the Teams Group. However, another group member is now able to e-mail the external user a link to the Teams Group. The external user can use this link to access the Group within the Teams app.

User is now in Teams Group

NOTE: Keep in mind that I am only using the Teams Group as an example. You can use somewhat similar techniques to provide access to a variety of Microsoft365 AND Azure AD content.

MORE M365 CONTENT TO COME!
POSITIVE ATTITUDE = POSITIVE RESULTS

REFERENCES:
How To Enable Guest Access for Office 365

Author Lance LingerfeltPosted on

Windows Server Core – How to have PowerShell automatically start when logging onto the session.

In my environment, I have a Windows Server (2019) Core edition server installed with Exchange 2019. Most of the time, I have to get on the server to run PowerShell commands for maintenance purposes, etc…

Well, by default, Windows Server Core opens the command prompt when you logon and then I have to manually open PowerShell from there to run cmdlets, etc…

However, if you would like to change the default cmd to PowerShell, you can change it by changing the Registry value.

The Registry that I’m talking is located under the following location:

Change the Shell Value in the Registry

The easiest way I see to change the value is to use the Set-ItemProperty cmdlet within PowerShell.

Open Windows PowerShell within Server Core command prompt. You can type “PowerShell” on your command prompt.

Then, enter the following command on PowerShell console and hit enter:

Once completed, you will need to reboot the computer from PowerShell:

When the computer has rebooted and you have logged on, PowerShell should load by default instead of Command Prompt.

EVEN MORE INFORMATION

Now, since I have an Exchange Server installed on this server, there is a Command in the $bin directory called LaunchEMS.cmd that will load the Exchange Management Shell for you. So instead of loading just PowerShell, I tell WinLogon to load Exchange Management Shell so that I do not have to do any additional typing or searching for EMS on the box. Remember, Server Core has no GUI!

I run the same commands as above, but just change the value to LaunchEMS.cmd

Then Restart the Computer:

Once Rebooted, you can logon and EMS will be the only window prompt that loads in the shell!

Exchange Management Shell loads when you logon

NOTE: You can always run cmd from the prompt to open Command Prompt and also run PowerShell.exe to open regular PowerShell from the EMS Session Window.

REMAIN POSITIVE!
THANKS FOR READING!

REFERENCES:
Windows Server Core: How to start PowerShell by Default

Author Lance LingerfeltPosted on

Exchange Server Quarterly Update – Cumulative Update 17 for Exchange Server 2016

As many of you that follow Exchange have knowledge of, Microsoft releases their updates for Exchange Server every 3 months. Below is the latest update for Exchange Server 2016. I do not run 2016 any longer in my lab, but please post if you have issues with the installation and I can investigate!

Cumulative Update 17 for Microsoft Exchange Server 2016 was released on June 16, 2020. This cumulative update includes fixes for nonsecurity issues and all previously released fixes for security and nonsecurity issues. These fixes will also be included in later cumulative updates for Exchange Server 2016.  

This update also includes new daylight saving time (DST) updates for Exchange Server 2016. For more information about DST, see Daylight Saving Time Help and Support Center.

Known issues in this cumulative update

  • In multidomain Active Directory forests in which Exchange is installed or has been prepared previously by using the /PrepareDomain option in Setup, this action must be completed after the /PrepareAD command for this cumulative update has been completed and the changes are replicated to all domains. Setup will try to run the /PrepareAD command during the first server installation. Installation will finish only if the user who initiated Setup has the appropriate permissions.

    Notes
    • If you are upgrading from Cumulative Update 13 for Exchange Server 2016 or a later cumulative update for Exchange Server 2016 to Cumulative Update 17 for Exchange Server 2016, then there’s no need to run the /PrepareAD or /PrepareDomain. No additional actions (prepareAD, prepareDomain, or assigning permissions) are required.
    • If you have ever skipped a Cumulative Update (for example, you are upgrading from an earlier version before Cumulative Update 13 for Exchange Server 2016), or this is a first Exchange Server installation in the AD, then this Known Issue section should be taken care of.
      • About the /PrepareDomain operation in multidomain:

        The /PrepareDomain operation automatically runs in the Active Directory domain in which the /PrepareAD commandis run. However, it may be unable to update other domains in the forest. Therefore, a domain administrator should run the /PrepareDomain in other domains in the forest.
      • About the permission question:

        As the /PrepareAD is triggered in Setup, if the user who initiates Setup isn’t a member of Schema Admins and Enterprise Admins, the readiness check will fail and you receive the following error messages.

        the Active Directory schema isn't up-to-date error

        To avoid the errors, either the user should join Schema Admins and Enterprise Admins groups or another user in Schema Admins and Enterprise Admins groups manually runs the /PrepareAD for this Cumulative Update first. Then the Exchange admin user can start Setup.
  • Autodiscover Event ID 1 occurs after you install Cumulative Update 14 for Exchange Server 2016. For more information, see KB 4532190.

Issues that this cumulative update fixes

This cumulative update fixes the issues that are described in the following Microsoft Knowledge Base articles: 

  • 4559444 Conversion from HTML to RTF removes non-breaking space in Exchange Server 2016
  • 4559435 Introduce an OrganizationConfig flag to enable or disable recipient read session in Exchange Server 2016
  • 4547707 Enable piping for Restore-RecoverableItems in Exchange Server 2019 and 2016
  • 4559436 Attachments with properties (like Azure Information Protection labels) don’t always match in Exchange Server 2016
  • 4559437 PR_RECIPIENT_ENTRYID is computed if no email address or type in Exchange Server 2016
  • 4559438 Edge Transport server hangs in Exchange Server 2016
  • 4559439 EAS creates failure report if a message with unknown recipients is in Drafts in Exchange Server 2016
  • 4559440 Export to a PST for an eDiscovery search fails in Exchange Server 2016
  • 4559441 Foreign language characters set in RejectMessageReasonText of a transport rule aren’t shown correctly in Exchange Server 2016
  • 4559442 2080 Events caused by empty values in HKLM\SYSTEM\CurrentControlSet\Services\MSExchange ADAccess\Instance0 in Exchange Server 2016
  • 4549689 HMA EvoSTS certificate rollover causes authentication prompts due to stalled key on worker process spawn (warmup phase) in Exchange Server 2016
  • 4559443 Managed Folder Assistant fails with Event ID 9004 NotInBagPropertyErrorException in Exchange Server 2016
  • 4559446 Changes to Outlook on the web blocked file extensions and MIME types in Exchange Server 2016

Get Cumulative Update 17 for Exchange Server 2016

Download Center

Download Download Cumulative Update 17 for Exchange Server 2016? (KB4556415) now

Download Download Exchange Server 2016? CU17 UM Language Packs now

Notes

  • The Cumulative Update 17 package can be used to run a new installation of Exchange Server 2016 or to upgrade an existing Exchange Server 2016 installation to Cumulative Update 17.
  • You don’t have to install any previously released Exchange Server 2016 cumulative updates or service packs before you install Cumulative Update 17.

Cumulative update information

Prerequisites

This cumulative update requires Microsoft .NET Framework 4.8.

A component that’s used within Exchange Server requires a new Visual C++ component to be installed together with Exchange Server. This prerequisite can be downloaded at Visual C++ Redistributable Packages for Visual Studio 2013. For more information, see KB 4295081.

For more information about the prerequisites to set up Exchange Server 2016, see Exchange 2016 prerequisites.

Restart requirement

You may have to restart the computer after you apply this cumulative update package.

Registry information

You don’t have to make any changes to the registry after you apply this cumulative update package.

Removal information

After you install this cumulative update package, you can’t uninstall the package to revert to an earlier version of Exchange Server 2016. If you uninstall this cumulative update package, Exchange Server 2016 is removed from the server.

CHECK FOR UPDATES REGULARLY!
POSITIVE ATTITUDE
EQUALS
POSITIVE MINDSET!

REFERENCES:
Exchange Server 2016 CU17

Author Lance LingerfeltPosted on

Exchange Server Quarterly Update – Cumulative Update 6 for Exchange Server 2019

As many of you that follow Exchange have knowledge of, Microsoft releases their updates for Exchange Server every 3 months. Below is the latest update for Exchange Server 2019. I will be installing it very soon and will post any issues I may have with Server Core and GUI versions of Windows 2019.

Cumulative Update 6 for Microsoft Exchange Server 2019 was released on June 16, 2020. This cumulative update is a security update. It includes fixes for nonsecurity issues and all previously released fixes for security and nonsecurity issues. These fixes will also be included in later cumulative updates for Exchange Server 2019. 

This update also includes new daylight saving time (DST) updates for Exchange Server 2019. For more information about DST, see Daylight Saving Time Help and Support Center.

Known issues in this cumulative update

  • In multidomain Active Directory forests in which Exchange is installed or has been prepared previously by using the /PrepareDomain option in Setup, this action must be completed after the /PrepareAD command for this cumulative update has been completed and the changes are replicated to all domains. Setup will try to run the /PrepareAD command during the first server installation. Installation will finish only if the user who initiated Setup has the appropriate permissions.

    Notes
    • If you are upgrading from Cumulative Update 2 for Exchange Server 2019 or a later cumulative update for Exchange Server 2019 to Cumulative Update 6 for Exchange Server 2019, there’s no need to run the /PrepareAD or /PrepareDomain. No additional actions (prepareAD, prepareDomain, or assigning permissions) are required.
    • If you have ever skipped a Cumulative Update (for example, you are upgrading from an earlier version before Cumulative Update 2 for Exchange Server 2019), or this is a first Exchange Server installation in the AD, then this Known Issue section should be taken care of.
      • About the /PrepareDomain operation in multidomain:

        The /PrepareDomain operation automatically runs in the Active Directory domain in which the /PrepareAD commandis run. However, it may be unable to update other domains in the forest. Therefore, a domain administrator should run the /PrepareDomain in other domains in the forest.
      • About the permission question:

        As the /PrepareAD is triggered in Setup, if the user who initiates Setup isn’t a member of Schema Admins and Enterprise Admins, the readiness check will fail and you receive the following error messages.

        the Active Directory schema isn't up-to-date error

        To avoid the errors, either the user should join Schema Admins and Enterprise Admins groups or another user in Schema Admins and Enterprise Admins groups manually runs the /PrepareAD for this Cumulative Update first. Then, the Exchange admin user can start Setup.
    • Autodiscover Event ID 1 occurs after you install Cumulative Update 3 for Exchange Server 2019. For more information, see KB 4532190.

Issues that this cumulative update fixes

This cumulative update also fixes the issues that are described in the following Microsoft Knowledge Base articles:

  • 4559441 Foreign language characters set in RejectMessageReasonText of a transport rule aren’t shown correctly in Exchange Server 2019
  • 4547707 Enable piping for Restore-RecoverableItems in Exchange Server 2019
  • 4549689 HMA EvoSTS certificate rollover causes authentication prompts due to stalled key on worker process spawn (warmup phase) in Exchange Server 2019
  • 4559446 Changes to Outlook on the web blocked file extensions and MIME types in Exchange Server 2019
  • 4559440 Export to a PST for an eDiscovery search fails Exchange Server 2019
  • 4559439 EAS creates failure report if a message with unknown recipients is in Drafts in Exchange Server 2019
  • 4559442 2080 Events caused by empty values in HKLM\SYSTEM\CurrentControlSet\Services\MSExchange ADAccess\Instance0 in Exchange Server 2019
  • 4559438 Edge Transport server hangs in Exchange Server 2019
  • 4559443 Managed Folder Assistant fails with Event ID 9004 NotInBagPropertyErrorException in Exchange Server 2019
  • 4559437 PR_RECIPIENT_ENTRYID is computed if no email address or type in Exchange Server 2019
  • 4559444 Conversion from HTML to RTF removes non-breaking space in Exchange Server 2019
  • 4559436 Attachments with properties (like Azure Information Protection labels) not always matching in Exchange Server 2019
  • 4559435 Introduce an OrganizationConfig flag to enable or disable recipient read session in Exchange Server 2019

Get Cumulative Update 6 for Exchange Server 2019

Volume Licensing Center

To get Cumulative Update 6 for Exchange Server 2019, go to Microsoft Volume Licensing Center.

Note The Cumulative Update 6 package can be used to run a new installation of Exchange Server 2019 or to upgrade an existing Exchange Server 2019 installation to Cumulative Update 6.

Cumulative update information

Prerequisites

This cumulative update requires Microsoft .NET Framework 4.8.

A component that’s used within Exchange Server requires a new Visual C++ component to be installed together with Exchange Server. This prerequisite can be downloaded at Visual C++ Redistributable Package for Visual Studio 2012.

For more information about the prerequisites to set up Exchange Server 2019, see Exchange 2019 prerequisites.

Restart requirement

You may have to restart the computer after you apply this cumulative update package.

Registry information

You don’t have to make any changes to the registry after you apply this cumulative update package.

Removal information

After you install this cumulative update package, you can’t uninstall the package to revert to an earlier version of Exchange Server 2019. If you uninstall this cumulative update package, Exchange Server 2019 is removed from the server.

CHECK FOR UPDATES REGULARLY!
POSITIVE ATTITUDE
EQUALS
POSITIVE MINDSET!

REFERENCES:
Exchange Server 2019 CU6

Author Lance LingerfeltPosted on

Lifting EWS Throttling for Exchange Online Mailbox Migrations now in the M365 Support Assistant

In my experience with doing Exchange migrations to the cloud, there always seemed to be an issue with EWS throttling causing very slow mailbox moves. The remediation to this was to always contact Microsoft EXO Support and open a ticket to request that your EWS throttling for your tenant be lifted so you could move your mailboxes more quickly.

The reason for the EWS Throttling was to keep large amounts of data from flooding the front end server farm possibly causing a temporary outage or corruption of data going to the cloud since there are literally thousands of customers going through the same server farm and possibly migrations. Throttling was a way to keep everything in check.

Microsoft recently made an interesting change to the automated support handling capabilities of the Microsoft 365 admin center to handle requests for Exchange Web Services (EWS) throttling to be lifted for up to 90 days without human intervention.

Here is how to request the throttling be lifted via the M365 Support Assistant:

  • Go to the Help (?) section of the Microsoft 365 admin center.
  • Click the Need Help icon.
  • Enter “EWS throttling” as the search phrase.
  • Click Run tests when asked to check your environment. Essentially, the tests check what EWS throttling applies to the tenant.
Getting EWS Throttling Support
Running the Tests from the Admin Center
  • The support assistant checks the tenant settings and concludes that EWS is throttled (the normal situation). You’ll then be offered the chance to update the settings to the tenant EWS policy to lift throttling for 30, 60, or 90 days.
  • Select the number of days you’d like to adjust the policy for and then Update Settings.
  • After a short delay, the support assistant should confirm that the settings have been changed.
Select the number of days to lift throttling and click Update Settings

Once the setting is changed in the tenant, it will be effective after about 15 minutes for replication to the server farm. You should then be able to run your migrations at full speed.

Settings Changed Successfully

NOTE: Changing this setting is only effective for EWS migrations and NOT IMAP Migrations (such as from G-Mail).

Having this option online in the portal saves a tremendous amount of time when working to get your Exchange migrations to the cloud completed. Hopefully there will be more to come. I just hope it doesn’t keep me out of a job!

I AM CURRENTLY LOOKING FOR NEW OPPORTUNITES!
POSITIVE MINDSET, POSITIVE ATTITUDE!

REFERENCES:
Microsoft Automates EWS Throttling

Author Lance LingerfeltPosted on

Troubleshooting OOF/OOO (Out of Office) Replies in Exchange and Exchange Online

Since the COVID-19 virus has managed to get me laid-off and not working, I have not had too much to post in the past months. And although it has wavered greatly, I will remain positive and hope that someone will see the value that I can bring to their organization in the near future.

With that said, I wanted to repost this article that was sent to me as this issue arises quite often in the workplace when someone has left for vacation and is not in the office. So, here it is:

Understanding and troubleshooting Out of Office (OOF) replies

Out of Office replies can sometimes be a bit of a mystery to people; how do they work? What do you do if they don’t work? In this blog post, we will discuss the bits and pieces of Out of Office and some of the main reasons why an Out of Office (aka. OOF) reply might not get delivered to users. Note that while we are writing this from the viewpoint of an Exchange Online configuration, many of same things can be applied to on-premises configuration also. By the way – did you ever wonder why “OOF” is used instead of “OOO”? If you did, see this!

What is an Out of Office reply?

Out of Office replies are also known as OOF (or OOO) replies or automatic replies. They are Inbox rules that are set in the user’s mailbox by the client. OOF rules are server-side rules, so the response is sent regardless of whether a client is running, or not.

There are several ways of setting up automatic replies:

First, it can be set up as an automatic reply feature from Outlook, like this. It can also be configured using other clients, such as Outlook on the web (OWA), PowerShell command (Set-MailboxAutoReplyConfiguration). Admins can set up OOF replies on behalf of (forgetful) users from the M365 Admin Portal.

Other than using built-in OOF functionality, another thing people sometimes do is use rules to create an Out of Office message while they are away.

By design, Exchange Online Protection uses the high risk delivery pool (HRDP) to send the out of office replies, because they are lower priority messages.

Types of OOF rules

There are three types of OOF rules: Internal, External and Known Senders (Contact list). They are stored in the mailbox with the names in the following table. If a mailbox has set only Internal OOF, there will be no external rule created and the mailbox will have one OOF rule.

TypeMessage ClassPR_RULE_MSG_NAME
InternalIPM.Rule.Version2.MessageMicrosoft.Exchange.OOF.KnownExternalSenders.Global
ExternalIPM.Rule.Version2.MessageMicrosoft.Exchange.OOF.AllExternalSenders.Global
Known ExternalIPM.ExtendedRule.MessageMicrosoft.Exchange.OOF.KnownExternalSenders.Global

Note: Apart from OOF rule, other rules like the Junk Email rule will also have the “IPM.ExtendedRule.Message”  message class; the MSG_NAME will determine what the rule is for.

OOF rule details

All Inbox rules can be viewed using MFCMapi tool:

Logon > profile that you are accessing > Top of information store > Inbox > right click ‘Open associated contents table’. They are listed under the Message Class column. All Inbox rules will have the same message class IPM.Rule.Version2.Message and there is one message class and name for each Inbox rule.

For all rules, the name of the rule is in stored in the PR_RULE_MSG_NAME property. So, if there are 4 Inbox rules, there will be 4 IPM.Rule.Version2.Message one for each rule, and the name of the rule is stored in PR_RULE_MSG_NAME.

OOF rules in MFCMapi:

OOF01.jpg

And OOF rule templates in MFCMapi:

OOF02.jpg

History of OOF replies

OOF response is sent once per recipient. Recipients to whom the OOF was sent are stored in the OOF history and are cleared out when the OOF state changes (enabled/disabled) or the OOF rule is modified. OOF history is stored in the user’s mailbox and can be seen using MFCMapi tool at: Freebusy Data > PR_DELEGATED_BY_RULE.

OOF03.jpg

Note: If you want to send response to the sender every time instead of just once, you can apply mailbox server side rule “have server reply using a specific message” to send automatic reply instead of using the OOF rule. This server-side rule will send reply to the sender every time a message is received.

Now that we know what OOF replies are and how they are stored on the server, we can move on to address some of the scenarios where OOF is not sent to the sender. We will also discuss possible fixes and some more frequently seen issues you may have with OOF configuration.

The first category of issues we will talk about are issues related to OOF replies not being received by the sender of the original message.

OOF issues related to transport rules

When OOF doesn’t seem to be sent for all users in the tenant, usually there is a transport rule causing the issue. Check all the transport rules that may apply to the affected mailbox using step two of this article.

If you suspect a delivery problem, run a message trace from the Office 365 tenant.  We know that OOF response is sent back to the original sender of the message, so for OOF messages, the sender of the original message becomes the recipient when tracking. We should then be able to tell if OOF reply has been triggered and sent to external or internal recipient. If a Transport rule is blocking the OOF response, the message trace will clearly show you that.

There is one scenario I would like to highlight when it comes to transport rules blocking OOF replies. Let’s assume that you moved the MX record to a 3rd party anti-spam solution; you have created a transport rule to reject any email coming from any other IP address than the 3rd party anti-spam.

The transport rule will look something like this:

Description:
If the message: Is received from ‘Outside the organization’ Take the following actions: reject the message and include the explanation ‘You are not permitted to bypass the MX record!’ with the status code: ‘5.7.1’ Except if the message: sender ip addresses belong to one of these ranges: ‘1xx.1xx.7x.3x’

ManuallyModified: False

SenderAddressLocation: Envelope

As OOF replies have a blank (<>) Return-Path, you will see that the rule is unexpectedly matching the transport rule and the OOF responses are getting blocked.

In order to fix this, you can change the transport rule property of ‘Match sender address in message’ to ‘Header or envelope’, so the checks will also be done against ‘From’ (also known as the ‘Header From’ address), ‘Sender’ or ‘reply-to’ fields. More information about the mail flow rule conditions is here.

OOF04.jpg

JournalingReportNdrTo mailbox setting

If the affected mailbox is the one which is configured under JournalingReportNdrTo, OOF replies will not be sent for that mailbox. Moreover, journaling emails may be affected as well. It is recommended to create a dedicated mailbox for the JournalingReportNdrTo setting. Alternatively, you can set it to an external address.

For more details on how to solve this, please see this KB Article.

Forwarding SMTP address is enabled on the mailbox

If the affected user mailbox has SMTP forwarding enabled, OOF replies won’t be generated. This can be checked in user mailbox settings (OWA):

OOF05.jpg

In PowerShell:

OOF06.jpg

Or, in the M365 Portal, user properties:

OOF07.jpg

Please follow the action on step one of this article for more information.

The type of the OOF reply set on remote domains

Remote domains offer you (among other settings) the opportunity to set the type of OOF reply that can be sent to users.

These types are the following:

  • External
  • ExternalLegacy
  • InternalLegacy
  • None

For more information about each of these OOF types, please refer to AllowedOOFType parameter in our Set-Remotedomain document.

The Out of Office type can be checked from Exchange Admin Center > Mail flow > Remote domains

OOF08.jpg

Or, with PowerShell:

OOF12.jpg

You need pay attention to what OOF type you have set up, as this will impact the OOF response and OOF may not be generated at all if the configuration is incorrect. Let’s assume you have a hybrid organization with mailboxes hosted both in Exchange on-premises and Exchange Online. In this scenario, by design only external messages will be sent to on-premises while AllowedOOFType is set to External. To be able to send internal OOF messages to on-premises in hybrid environment, you need to set the AllowedOOFType to InternalLegacy.

You also have option to send external Out of Office replies only for contacts at the mailbox configuration level (ExternalAudience: Known). This can make automatic replies not being sent to anyone external but contacts. The command to check the configuration is:

OOF10.jpg

Remote domain blocking OOF replies

Another setting on remote domains is one which lets you dictate whether you allow or prevent messages that are automatic replies from client email programs in your organization.

This can be found in Exchange Admin Center > Mail flow > Remote domains

OOF11.jpg

Or by running this PowerShell cmdlet:

OOF12.jpg

Note: If this option is set to false, no automatic replies will be sent to users for that domain. This setting takes precedence over the automatic replies set up at the mailbox level or over the OOF type (discussed above).

Please, keep in mind that $false is the default value for new remote domains that you create as well as the built-in remote domain named Default in Exchange Online.

If the email was marked as spam and sent to junk, an automatic reply will not be generated at all.

Pretty self-explanatory, that one!

Message trace shows delivery failure

If you investigate an OOF reply issue and in the message trace you find the following error message:

“550 5.7.750 Service unavailable. Client blocked from sending from unregistered domains.”

You should reach out to Support to find out why the unregistered domain block was enforced.

There are some other scenarios that might come up when working with OOF replies, let’s cover those next!

An old or duplicate OOF message is sent

This is likely due to a duplicate Inbox rule or the OOF history limit. The OOF history has a limit of 10,000 entries, if this threshold is hit, OOF will continue to be sent to recipients that are not already in the list as any new users can’t be added to the list. All users already in the list will not receive duplicate OOF replies. For more information you may want to check this article or follow the action plan below.

  • Remove the OOF rules and the OOF rule templates from the mailbox. To locate the rules and delete them go back to > “Inbox OOF rules”
  • Disable and then re-enable the OOF feature for the mailbox

Now, you can check again whether the OOF feature works as expected and symptoms do not occur.

Automatic replies cannot be enabled; an error message is received

While attempting to access automatic replies from the Outlook client, an error message is received saying that “Your automatic reply settings cannot be displayed because the server is currently unavailable. Try again later.”

To narrow down this issue, you should perform the following steps:

  • Confirm EWS protocol is enabled on the mailbox as OOF replies rely on it to be enabled (note that re-enabling this might take several hours to take effect)
  • Enable the OOF feature by using the following command:Set-MailboxAutoReplyConfiguration <identity> -AutoReplyState Enabled
  • Check whether the OOF feature works as expected.
  • If the issue is still there, review the rules quota on the mailbox: Get-mailbox -identity <mailbox> | fl RulesQuota
OOF13.jpg

By default, the RulesQuota has a maximum quota which is calculated by the size of the rules (not the number of rules). Maximum is 256 KB (262,144 bytes).

  • Remove the OOF rules and the OOF rule templates from the mailbox. To locate the rules and delete them go back to > “Inbox OOF rules”
    After you remove them, you can re-enable the OOF feature and then test again.

An automatic reply is still sent despite OOF being disabled

We have encountered scenarios in which OOF messages are still being sent, although it is disabled. Most of the time, we found that the rule is created manually by the end users using the out-of-office template.

So, as you can see, there is quite a bit that goes into troubleshooting OOF Replies and it is not all straight forward.

THANKS FOR READING!
I’M AVAILABLE FOR WORK!
PLEASE CONTACT ME FOR AN APPOINTMENT!

REFERENCES:
Troubleshooting OOF Replies (Exchange Team Blog)

Author Lance LingerfeltPosted on

PowerShell – How to create a custom view for your PS Output Objects

I have been doing some training on PowerShell Scripting this week and am going to be posting a number of articles on what I have been training on. This article deal with formatting your data output from your custom script or function to be viewed the way that you want it.
Have you ever run a cmdlet where the column width is not wide enough and your data get’s truncated? Well, here is a method that you can use to make the default output of your function or script display how you want it to.

The best way to do this is by using an existing xml formatting file as a template. Run the following PowerShell commands to access those templates:

Note: your custom view xml file will need to have the “.ps1xml” extension, which indicates that it is a “Windows Powershell XML Document”.

Now to creating the custom template. Let’s say you have the following function.

Now, create a custom view using the following file as a template:

C:\WINDOWS\system32\WindowsPowerShell\v1.0\DotNetTypes.format.ps1xml

Note: You can get a list of the format type template files that already comes with PS running the following command:

Sample Output:

Mode LastWriteTime Length Name
—- ————- —— —-
-a— 10/06/2009 21:41 27338 Certificate.format.ps1xml
-a— 10/06/2009 21:41 27106 Diagnostics.Format.ps1xml
-a— 23/07/2012 19:12 144442 DotNetTypes.format.ps1xml
-a— 23/07/2012 19:12 14502 Event.Format.ps1xml
-a— 23/07/2012 19:12 21293 FileSystem.format.ps1xml
-a— 23/07/2012 19:12 287938 Help.format.ps1xml
-a— 23/07/2012 19:12 97880 HelpV3.format.ps1xml
-a— 23/07/2012 19:12 101824 PowerShellCore.format.ps1xml
-a— 10/06/2009 21:41 18612 PowerShellTrace.format.ps1xml
-a— 23/07/2012 19:12 13659 Registry.format.ps1xml
-a— 23/07/2012 19:12 17731 WSMan.Format.ps1xml

Remember: A custom view must always end with “.format.ps1xml”

We will use DotNetTypes.format.ps1xml as a template. Using this file, create a file called “hrmctools.formatps1xml”. That file will contain the following information modified from the template:

SampleCustomXMLDownload

NOTE: In PS, the content of xml files are always CASE SENSITIVE!!!

Once you have created your own custom view you then need to tell PowerShell to apply the formatting by using the cmdlet Update-FormatData:

Once you run the above command, this custom format you created should now be loaded into memory. You can verify this by using the Get-FormatData cmdlet:

If you have not done so in your function or script, you can attach your custom view to your function or script using the “insert” method:

$MyObject.PSObject.TypeNames.Insert(0,’hrmctoolcustomformat’)

Note*: This code has already been inserted into our function listed in this example.
NOTE**:The first parameter “0” is something that you always type in. You can now confirm that the object has successfully been attached to the custom view, by typing in PowerShell:

Also if you output your object, you should now notice that its appearance should have now changed to those that you defined in the custom view.

A “Typename” is essentially a name that you give to your object. It tells PowerShell the type of object that it is. Know that it is possible for a number of commands to have outputting objects of the same type (the same typename value). This could affect other cmdlets and functions in your script, so be sure to debug if necessary.

HAPPY SCRIPTING!
MORE TO COME!!

REFERENCES:
Creating Custom Format Views

Author Lance LingerfeltPosted on

COVID-19 Update from LDLNET LLC

Message of Support

It has always been my goal to provide your company with sthe best IT solutions and I will continue to do so as effectively as I can. 

I am aware that the recent spread of COVID-19 may have you concerned. At LDLNET LLC, I take the safety and well-being of my present and future customers seriously.

What I Am Doing

To further prevent the spread of COVID-19, I am committed to enacting best practices laid out by the Center for Disease Control (CDC). To ensure the health and safety of your offices, I clean, sanitize and disinfect my equipment with EPA approved sanitizer and disinfectant between appointments. I can also provide 100% remote work to you company provided that I have secure access.

How I Can Help 

With precautions taken, I will continue to stay open for business. I will continue to provide the best IT Solutions for your business that I can. Please feel free to contact me to set up an appointment for your next IT project!

I sincerely hope that you and your family are in good health. As our community works towards a solution, please take your own precautionary measures to reduce your own risk of illness. The simple act of washing your hands can act as an effective barrier against germs and bacteria. I want everyone in our community to stay safe.

If you want to schedule a time to talk, give me a call today at (844) 884-7838 to schedule an appointment or visit the website http://store.ldlnet.net for information. 

Sincerely,
Lance Lingerfelt 
Owner/Operator 
LDLNET LLC

Remember to clean and disinfect high-touch surfaces daily in all common areas.

Hand washing is one of the best ways to protect you and your family from getting sick.

If you show symptoms or are sick, it is recommended that you stay home.

Author Lance LingerfeltPosted on

Exchange Server Quarterly Updates March 2020

Released: March 2020 Quarterly Exchange Updates

Today Microsoft is announcing the availability of quarterly servicing cumulative updates for Exchange Server 2016 and 2019. These updates include fixes for customer reported issues as well as all previously released security updates. 

Personal Note: I was recently involved in a layoff at Microsoft in the Vendor PFE world. I am currently looking for new engagements.

Calculator Updates

This quarterly Exchange release includes an important update to the Exchange 2019 Sizing Calculator.  We’ve made improvements to the logic to detect whether a design is bound by mailbox size (capacity) or throughput (IOPs) which affects the maximum number of mailboxes a database will support.  Previous versions of the calculator produced incorrect results in some situations.

The Exchange team highly recommends using calculator version 10.4, included with the March 2020 quarterly CU release, to size Exchange Server 2019 deployments.

MCDB Configuration Issues

Cumulative Update 5 for Exchange Server 2019 also fixes an issue that can happen when you use the Manage-MetaCacheDatabase.ps1 script to enable MetaCacheDatabase (MCDB).

This issue occurred because of a change in behavior in Windows Server 2019 that caused Get-Disk to return all uninitialized discs within the Database Availability Group (DAG) or cluster. The script then incorrectly tried to format an SSD on another DAG member. We documented a workaround for CU4 here, but we’ve fixed it in CU5.

Online Mode Search Issues

Cumulative Update 5 for Exchange Server 2019 is also required to fix a known issue with partial word searches when the client is using Outlook in online mode.

Release Details

The KB articles that describe the fixes in each release and product downloads are available as follows:

Additional Information

Microsoft recommends all customers test the deployment of any update in their lab environment to determine the proper installation process for your production environment. For information on extending the schema and configuring Active Directory, please review the appropriate documentation.

Also, to prevent installation issues you should ensure that the Windows PowerShell Script Execution Policy is set to “Unrestricted” on the server being upgraded or installed. To verify the policy settings, run the Get-ExecutionPolicy cmdlet from PowerShell on the machine being upgraded. If the policies are NOT set to Unrestricted you should use the resolution steps in KB981474 to adjust the settings.

Reminder: Customers in hybrid deployments where Exchange is deployed on-premises and in the cloud, or who are using Exchange Online Archiving (EOA) with their on-premises Exchange deployment are required to deploy the currently supported cumulative update for the product version in use, e.g.,

2013 Cumulative Update 23
2016 Cumulative Update 16 or 15
2019 Cumulative Update 5 or 4.

For the latest information on Exchange Server and product announcements please see: 
What’s New in Exchange Server and Exchange Server Release Notes.

I AM STILL CURRENTLY LOOKING FOR A NEW PROJECT OR ASSIGNMENT!
THANKS FOR READING!

Author Lance LingerfeltPosted on

Exchange Server Security Update KB4540123 fails with 0x80070643

PLEASE READ THE ENTIRE POST

I had a failure on one of my two Exchange 2019 CU4 servers when installing the Security Update for them:

Exchange Server Security Update KB4540123 fails with 0x80070643

I could not restart the install as it would fail when getting to the services stoppage part of the installation. I saw this error in the ServiceControl.log file in the C:\ExchangeSetupLogs Directory

I had searched around based on the error code 0x80070643 and found these answers that some had used to get the installation to work.

LINK HERE

I downloaded the .msp file to install manually and read the answers on the web-page. It was said that there was an issue with the ServiceControl.ps1 file that is in the Exchange Server BIN directory when running Patch with the Verbose logging enabled:

I kept reading and digging into the fix for the file. It was said to modify a number of lines in the ServiceControl.ps1 script:

Once I made those changes. I renamed the original ServiceContol.ps1 to a .old file and saved this modified file to the BIN directory. I was then able to successfully run the Security Update.

BUT WHY DID THIS SERVER FAIL AND NOT THE OTHER?!?

Both are CU4, but the server that failed had been upgraded from CU1 where the one that did NOT fail was a clean installation of CU4. So, I checked the ServiceConrol.ps1 file on both servers:

Older Exchange Install
Date of ServiceControl.ps1 is 1/1/2020
CU4 Clean Installation
Date of ServiceControl.ps1 is 2/3/2020

I have not tested this, but maybe if I had copied the ServiceControl.ps1 file from the CU4 clean installation to the original install, the script might have worked since the creation dates are different and I have no other version information to go on. I will verify this though. For now, the changes to the script allowed me to successfully install the Security Update Successfully.

NOTE: All the Exchange and IIS Services were in a Startup Mode: Disabled state and I had to reset them ALL to Automatic. Once that was completed and the server rebooted, Exchange was returned to normal state. ****Also, just to be safe, I ran a great script that assures the server is out of maintenance mode. You can get the script HERE. You can also get the script to put the server into maintenance mode. You can get the script HERE. These scripts will work on Exchange 2013 and above servers.

UPDATE / 3/12/2020 2:39 AM EST

Something was still wrong with the server. ActiveSync and PAM started breaking. I started having all sorts of authentication problems. I decided to restore the server from backup from Tuesday. I forgot to backup the ServiceControl.ps1 file that modified, but I moved the one from the successful installation to the restored server and am currently running the update. I will see if it works and send the information to you.

UPDATE / 3/12/2020 3:45 AM EST

The restore worked great and placing the ServiceControl.ps1 file in the BIN directory on the prior failed Exchange Server did allow for the installation to complete successfully. I have tested ActiveSync and Authentication which is now functioning properly. Hooray!

SEND ME YOUR IDEAS/ FOR POSTS!
HAPPY TROUBLESHOOTING!

REFERENCES:
Exchange Server Security Update fails with 0x80070643
Exchange Maintenance Mode Script (Start)
Exchange Maintenance Mode Script (Stop)
Exchange Server 2019 CU4 Security Update

Author Lance LingerfeltPosted on

Exchange 2016 Deployment MUST READ Documentation

In my role as a PFE for Microsoft, I have been going through many deployments of Exchange 2016 from Exchange 2010 due to the end of life deadline for Exchange 2010. I am actually in the middle of three enterprise level deployments this month.

Because of this, I wanted to provide some MUST READ links with consideration to the Exchange 2016 Deployment process. Please click on the links below, and they will take you to the documentation needed when preparing for a Exchange 2016 migration and deployment from Exchange 2010.

IMPORTANT READ:
Exchange On-Premises Best Practices for migration from Exchange 2010 to Exchange 2016 (Lots of important links and articles within this document!)

Other relevant articles:
Exchange 2016 System Requirements
Exchange Deployment Site Consideration
What changes in AD when Exchange 2016 is installed
Exchange 2016 Schema Changes to AD
Exchange Server Virtualization
Exchange Server Preferred Architecture
Load Balancing in Exchange Server
Load Balancing Exchange 2016 In Depth

Hybrid Considerations:
Decommissioning Exchange 2010 servers in a Hybrid Deployment
How and when to decommission On-Prem Exchange Hybrid Servers
Hybrid Deployment Prerequisites

I will add more articles as they become relevant in my experiences with my customers and feel they could be relevant here as well. If you have a suggestion for a link that should be considered added to this post, feel free to leave a comment!

THANKS AGAIN FOR READING!
SUGGEST A LINK!
GOOD LUCK ON YOUR DEPLOYMENT!

Author Lance LingerfeltPosted on

Adding Windows Capability to Server Core to add features needed for Application Compatibility

I’ve been working on installing Windows Server 2019 Core into my network to be able to look at new features for Windows Administration and learning how Server Core works. I was able to install a virtual machine with Server Core and get it activated. I then wanted to place my custom PowerShell script for loading PowerShell into the Server Core Environment.

So, I added the Server Core Server to the Windows Admin Center and copied my custom scripts for PowerShell into the proper directory:

Windows Admin Center
Windows Admin Center

I then logged on remotely to the server and started PowerShell. When I did that, I got this error with the script load:

Error that IE First Run has not been completed
Error that IE First Run has not been completed

At first, I tried using the -UseBasicParsing as a switch to see if that would repair the issue in the script. It did not because, IE is not installed by default on the default installation of Server Core. That is so there is less of a footprint that can be attacked by a hacker. I needed this installed though so that the Invoke-WebRequest cmdlet would load my script parameters properly.

I started looking for answers to how to install IE onto the Server Core box and found the following article. I had to run the Add-WindowsCapability cmdlet on the server to install the optional components. When I did, I received an error:

Error when adding the Windows Capability
Error when adding the Windows Capability

So I found out that there is a block that WSUS does keeping the cmdlet from going to the online source to download the software package and producing this error. After researching, I found this article. I setup a Group Policy to make sure this setting is propagated to my Server Core machine. I also setup in the same policy the ability to turn off the First-Run for IE so that you do not get that message and have to open IE to “set it up”

Group Policy Setting with Path to Templates Specified
Group Policy Setting with Path to Templates Specified

I then ran a gpupdate /force on the Server and was able to download the components for IE and App Compatibility.

Successful Installation of Windows Capability
Successful Installation of Windows Capability

I then rebooted the server and now my PowerShell loads successfully:

Successful PowerShell Load
Successful PowerShell Load

I learned a few different new things here and was able to get Server Core working more the way that I like it. I will keep posting updates when I run into issues with this type of installation. I would definitely give the Windows Admin Center a try as it has more robust features than Server Manager has, especially for Server 2019 and Server Core.

CONQUER THE UNCOMFORTABLE TO GROW!
POSITIVE ATTITUDE ABIDES!

REFERENCES:
RSAT Tools Installation Error 0x800f0954 – Windows 10 1809
Server Core App Compatibility Feature on Demand (FOD)
“Set Up Internet Explorer 11” Bypass with GPO or Registry

Author Lance LingerfeltPosted on

How to address Federation Trust issues in Hybrid Configuration Wizard (HCW)

During my time as a PFE for Microsoft, I have encounted many issues with Federation in a Hybrid Exchange Deployment. Recently, the following support announcement came out and I wanted to share as I hope this can help others that may be having issues out there.

One of the more common causes of HCW failures is the Federation Trust step for the Exchange on-premises organizations in Full hybrid configurations (Classic or Modern topologies).

Federation trust is a mandatory step in the on-premises Exchange organizations when configuring Full hybrid deployments, as this allows us to create organization relationships (for features like hybrid free/busy or OWA/EAS redirection) and sharing policies (1:1 hybrid calendar sharing). In Exchange Online multi-tenant organizations, federation trust is already in place.

Below is an illustration of an Exchange hybrid deployment where both the Exchange on-premises organization and the Exchange Online organization have a trust with Azure Authentication System (formerly called Microsoft Federation Gateway):

Example of Hybrid Federation

Before getting to our subject, let’s quickly go over different hybrid configurations and Hybrid Configuration Wizard (HCW) – as this is the supported tool to configure hybrid deployments.
There are 2 flavors of hybrid configurations:
• Classic hybrid
• Modern hybrid

At this time, each of those supports the following hybrid modes:

  • Full
  • Minimal (which further breaks down into…)
    • Express (a one-time sync)
    • “Actual minimal”

A quick overview of Full / Minimal / Express options, can be found here. More info on HCW is here.

As mentioned earlier, a federation trust is created by HCW only in Full Hybrid.

HCW logs are located at %appdata%\Microsoft\Exchange Hybrid Configuration on the machine from where HCW was ran. The easiest way to get to them is to press F12 in the HCW window to open the Diagnostic tools and from there you can Open Folder Logging or Open Log File directly.

When you have issues with federation trust, the log will usually show errors when one of the following cmdlets are executed:

Set-FederationOrganizationIdentifier
or
Add-FederatedDomain (but can be other cmdlets as well).

Once you identified the exact cmdlet failing and where (Session=OnPremises – means Exchange Management Shell and Session=Tenant means Exchange Online PowerShell), you should copy-paste the failing command and try to execute it manually and see if that is failing as well (most likely it will). You can also open the shells from F12 Diagnostic tools windows in HCW.

In order to get more details on the error and to rule out this is not an issue with HCW itself, you will need to separately run the same command that threw exception in HCW log and add Verbose switch to get verbose details of the error and the serialized remote exception.

For example, if the Exchange server version is Exchange 2010, you will run the failing command with Verbose switch in Exchange Management Shell (EMS), see if that fails and then get the serialized remote exception.

Example:

If the Exchange Server version is Exchange 2013/2016 and the above commands didn’t show more details on the error, we can also try the following:

  • Open regular Windows PowerShell (blue background) on the Exchange Server 2013/2016
  • Run command: add-pssnapin exchange
  • Run command that gave error in HCW and add a Verbose switch

Example:

Once you’ve gathered the verbose error / serialized exception, try to understand where it is failing (or provide it to Microsoft Support together with the HCW log).

Common Errors with Remediation Steps

  • Federation trust fails with “Object reference not set to an instance of an object”

This is a known old issue on Exchange 2016 CU7 servers, make sure your Exchange servers are updated to the latest CU.

Full error in the HCW log:

Resolution: Install the latest CU for Exchange 2016

  • Federation fails with “Proof of domain ownership has failed”

Full error in the HCW log:

Resolution:

• Check the TXT record for your domain(s) in HCW log or in Exchange Management Shell with command Get-FederatedDomainProof -DomainName
• See if it matches your published TXT record with either nslookup utility or by checking internet websites like https://www.whatsmydns.net/ put your domain in hostnames, type=txt, Nameservers – Authoritative

You would look for errors, missing records or unusual formatting (characters, spaces, quotes, TXT record split in half).

  • Federation fails with “An unexpected error occurred on a receive” or “An unexpected error occurred on a send.”

Full error in the HCW log:

Resolution:

Check outbound access from all your Exchange Servers to Microsoft Federation Gateway by browsing using Internet Explorer with PSEXEC tool (with -s and -i switches) from the Exchange Server (this will use Internet Explorer under System Account / Exchange Server Account).

In this example, “Windows Live” is actually this exact URL: https://domains.live.com/service/managedelegation2.asmx

From on-premises Exchange to Office 365, the Exchange 2010 MBX & CAS or 2013 MBX (backend) or 2016 / 2019 would need outbound Internet access to the Microsoft Federation Gateway in addition to https://outlook.office365.com/ews/exchange.asmx

Verify the machine/system account can access these Microsoft Federation Gateway URLs:

For a complete list of O365 URL & IP addresses, see these articles:

Note: If the Exchange requires a proxy server to access the Internet, specify the proxy server using “Set-ExchangeServer myExchange01 -InternetWebProxy http://myproxy:80”. Notice such proxy can’t require any user authentication for outbound Internet access, and the proxy must start with HTTP: and not HTTPS: (secure SSL).

You can also set the proxy using netsh as well.

set proxy proxy-server=”http=myproxy;https=sproxy:88″ bypass-list=”*.contoso.com” 

In rare instances, you can use the machine/system account to access the URLs from the browser, but Exchange cmdlets still failed with “Could not establish trust relationship for the SSL/TLS secure channel.” If that happens, make sure the certificate authorities for the urls are installed at the Third-Party Root Certification Authorities of the machine local certificate location.

REFERENCE:
Netsh Commands for Windows Hypertext Transfer Protocol (WINHTTP)
Firewall Considerations for Federated Delegation 

Federated delegation features require that the Mailbox and Client Access servers in your organization have outbound access to the Internet by using HTTPS. You must allow outbound HTTPS access (port 443 for TCP) from all Exchange 2010 Mailbox and Client Access servers in the organization.

  • There is no specific error / exception, in HCW log you would see it stops without any specific error.

Full error in the HCW log:

Resolution:

Look for orphaned federation trust:

Get-FederatedOrganizationIdentifier | FL

or

in HCW log if you see something with “DEL“: “contoso.com/Configuration/Deleted Objects/Microsoft Federation Gateway/DEL: <xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx>”.

Solution is to remove the orphaned federation trust and re-run HCW.

Reference here.

NOTE: as a first step, you can try to run the command remove-federateddomain with the switch -Force. Also, you don’t need to recreate federation trust manually, just re-run HCW (this will recreate federation trust for us)

  • Federation Trust fails with “InternalError InternalError: Internal error.”.”.””

Full error in the HCW log:

Resolution:

Open request with Microsoft Support or check if any Service Incident is published. Please see this for more information.

  • Federation trust fails with “1007 Access Denied”

Full error in the HCW log:

Resolution:

“1007 Access Denied” error is usually when we have issues with:

  1. Windows Time on the Exchange Server. See this article or this article.
  2. Outdated federation trust (for example, federation trust certificate expired) and in this case you would remove federation trust by following these steps.

If the federation trust certificate is not found on any of the servers, then proceed with resolution from the next error.

As an example, from one HCW log, there seems to be this federation trust certificate expired on 05/13/2019:

  • Federation trust fails with “Federation Certificate cannot be found”

Full error in the HCW log:

Resolution:

Follow the procedure here to manually cleanup the federation trust from AD. Once this is done, re-run the HCW to re-create it automatically.

KEEP TROUBLESHOOTING!
REMAIN VIGILANT USING FOCUSED INTENT, NEVER EMOTIONALIZING (FINE)!

REFERENCE:
How To Address Federation Trust Issues using the Hybrid Configuration Wizard

Author Lance LingerfeltPosted on

Fixed | WSUS 2016/2019 | Error Code 0x8024401c | Windows 10 nor Windows Server 2016 reporting to WSUS

Problem:

I had recently had this error in WSUS where my Windows Server 2016 servers would NOT report into the WSUS Server. I would get an error stating 0x8024401c when manually performing a report now to the WSUS Server using:

Error from Windows Update on affected server

Solution:

Go to IIS Manager on the WSUS Server

Goto Advanced Settings of  WsusPool.

Make sure following settings are present/configured on the Pool, if not change it to below:

Make sure, the WSUS Entry in the Registry is having fully qualified domain name of WSUS Server.

NOTE: If you have Group Policy managing the WSUS Settings, then make sure you change the settings in the WSUS Policy to use the FQDN of the WSUS Server and run a gpupdate /force on the clients.

[image%5B2%5D]
Should be set to FQDN of your WSUS Server
i.e. “http://wsus.domain.com:8530”

Stop IIS on the WSUS Server

Edit the web.config located at following location on WSUS Server:

Replace the following lines in the config file and save in the same directory:

Restart IIS on the WSUS Server

Try updating the clients again. They should be able to report and update successfully.

HAPPY TROUBLESHOOTING!
POSITIVE OUTCOMES ARISE FROM POSITIVE ATTITUDES!

REFERENCES:
Fixed | WSUS 2016 | Error Code 0x8024401c | Windows 10 | Windows Server 2016

Author Lance LingerfeltPosted on

Exchange 2019 Setup Prerequisite Check fails for .NET 4.8 Framework in CU4 on Windows builds 1909 and 1903

Symptoms

When you deploy or upgrade to Microsoft Exchange 2019 Cumulative Update 4 (CU4) on Microsoft Windows Server 2019 or Windows 10 (Management Tools only) builds 1909 or 1903, the system prerequisites check fails, and you receive the following error message:

“This computer requires .NET Framework 4.8 (https://support.microsoft.com/kb/4503548).”

By default, Windows builds 1909 and 1903 already have .NET Framework 4.8 installed. When you try to reinstall the software, the installation fails.

Cause

This problem is caused by a prerequisite check that was introduced in Cumulative Update 4. This process checks incorrectly for .NET Framework 4.8. Because the prerequisite check doesn’t recognize that .NET Framework 4.8 is already installed.

Status

Microsoft is researching this problem and will post more information in this article when it becomes available.

Workaround

Important

Follow the steps in this section carefully. Serious problems might occur if you modify the registry incorrectly. Before you modify it, back up the registry for restoration in case problems occur.

  • Start “regedit.exe” as an administrator.
  • Locate the following registry subkey:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full

  • Right-click the subkey, and select Permissions from the shortcut menu.
  • Select Advanced.
  • In the Advanced Security Settings window, locate the Owner attribute at the top of the window.
  • Select Change next to the listed owner.
  • In the Enter the object name to select field, enter the name of the local administrator group. For example, enter <computername>\Administrator. Then, select OK.
  • In the Advanced Security Settings window, select the local administrator group that you changed ownership to, and then select Edit.
  • Change the basic permissions to Full Control.
  • Select OK three times to save the changes and return to the main Registry Editor window.
  • Locate the following key in the path:

Name: Release
Type: REG_DWORD
Data: 528040 (decimal)
Change the Data value to 528049 (decimal)

  • Rerun the Exchange system prerequisites check, and deploy or update Exchange Server 2019.
  • Start Registry Editor, locate the subkey that’s mentioned in step 2, repeat the necessary steps to locate the Release key, and then revert the Data value to 528040 (decimal).

This should allow everything to run correctly after the installation.

POSITIVE DAY TO YOU!
FINE = Focused Intent Not Emotionalizing

REFERENCES
Exchange CU4 Prerequisite Check Fails for .NET 4.8 for Windows Server builds 1909 and 1903