Tag: Script

Author Lance LingerfeltPosted on

Exchange Server Client Access URL Configuration Script

In my career, I have to be able to be efficient as most of my projects are on a time crunch schedule. Being able to quickly configure Exchange when setting up a server environment is crucial to the success of the project.

While still honing my skills in PowerShell, I was attempting to create my own script to help configure all of the Virtual Directories in one shot rather than go to each setting and configure them manually. It did not go very well, so as I do, I research and find great professionals that do great work in scripting so that I may learn from them.

In doing so, I found Paul Cunningham’s script that performs this. I took the following script and modified it to add the PowerShell Virtual Directory to it as I like to configure that as well.

***YOU CAN REM THE LINES OUT SHOULD YOU NOT WANT TO CONFIGURE THAT DIRECTORY***

Here is my version of the script:

NOTES:

  • PowerShell script to configure the Client Access server URLs for Microsoft Exchange Server 2013/2016. All Client Access server URLs will be set to the same namespace.
  • If you are using separate namespaces for each CAS service this script will not handle that.
  • The script sets Outlook Anywhere to use NTLM with SSL required by default.
  • If you have different auth requirements for Outlook Anywhere use the optional parameters to set those.
  • The script sets PowerShell to use Basic with SSL required by default.
  • If you have different authentication requirements for PowerShell use the optional parameters to set those.
  • PowerShell was added to the settings. Please be sure to REM those lines of code should you NOT want to configure the PowerShell Virtual Directory.

USAGE:

HAPPY SCRIPTING!
POSITIVE ENERGY!
PLEASE COMMENT!

REFERENCES:
Exchange Server Client Access URL Configuration Script
PowerShell Script to Configure Exchange Server Client Access URLs

Author Lance LingerfeltPosted on

Importing User Photos to Office 365 in bulk for your company.

In a previous post, I showed how you could update one user’s photo for their Outlook and AD profiles via PowerShell. In this post, we will explore how to do this for your entire organization via PowerShell to Office365.

NOTE: I have not tested the scripts as I do not have enough mailboxes in my O365 tenant along with not using a ‘.’ in my alias. If the scripts are incorrect, please inform me with the correction and I will update accordingly.

Please make sure that your photos are reviewed before posting, and try to keep the file size of the photos to a minimum. In Office 365, there exists a limitation for the user photo not to be more than 10 KB in size, but I will show you how to get around that limitation.

Having a user photo for each of your users is very beneficial as it personalizes each account to a face in the company. The user photos can be viewed in below locations:

  • Outlook Web Access
  • Contact Card
  • Thumbnail in emails
  • Outlook Client
  • Yammer
  • Lync Client
  • SharePoint (People Search / Newsfeed)

Steps to take:

  1. Remove the 10KB photo size limitation in Exchange Online
  2. Prepare a folder with all users photos
  3. Update the profile photos via a PowerShell cmdlet.

Connect to Exchange Online with the RPS Proxy Method to remove the 10K size limitation

NOTE: In the PowerShell cmdlet above, we connected using a different proxy method. This was to overwrite the limitation of uploading the images with size more than 10KB. Using the different proxy method (/?proxyMethod=RPS ) to connect to Office 365 in the above cmdlet accomplishes this.

Prepare a folder locally and place all the photos in that folder

Create a folder named C:/UserPics and make the filename of each photo be the username of that particular user. (i.e. llingerfelt.png)
The below script should be able account for aliases that have a ‘.’ in the id as well. (i.e. lance.lingerfelt)

NOTE: From my research, there is no set photo type that is required for the photo. My suggestion would be to keep the photos .png for size constraints while maintaining picture clarity.

Update the profile pictures via PowerShell

Create the following script and name it Photos-Update.ps1

Run Photos-Update.ps1 and the script should upload the photos to Office 365 and apply each photo to the corresponding user.

NOTE: If you’re still having some issues with the alias having a ‘.’ in the name, you can also configure the Photos-Update.ps1 script in this manner to get that working properly:

HAPPY SCRIPTING!
PLEASE COMMENT!

store.ldlnet.net
LDLNET LLC! Your Source for Pofessional IT Services!
www.servermonkey.com
Contact ServerMonkey.com for your IT Hardware Needs!

REFERENCES:
How to import Office365 User photos over 10KB & without CSV in bulk

Author Lance LingerfeltPosted on

Purging Soft Deleted mailboxes from Exchange Server

If you’re a seasoned administrator, you have knowledge that in Exchange, the database settings will allow you to set the deleted mailbox retention. The default is 30 days, but sometimes you need to purge all those deleted mailboxes to do some ‘spring cleaning’ as it were. Note that doing these cmdlets does not change the ‘Whitespace’ of the database or the size. In my case, I had to purge everything of a toxic individual that was tainting my network much to my disappointment and did the following to complete that task.

The following cmdlet will seek all Soft Deleted mailboxes within the database you select and manually purge them from Exchange.

Now, should you only want to remove one mailbox, you will need to get the GUID of that Soft Deleted mailbox first so that you can enter it for the identity parameter.

You can also preform a similar task for a disabled mailbox:

You can perform the task on all disabled mailboxes for that database as well:

NOTE: I would be very careful when performing either of these cmdlets as they will completely purge the mailboxes from the schema. If these cmdlets assist you with your ‘spring cleaning’, I will have been happy to assist.

HAPPY PURGING!
PLEASE COMMENT!
IGNORANCE IS NOT BLISS!

References:
Purging Deleted Mailboxes on Exchange 2013

Server Monkey
LDLNET LLC’s Preferred Server Equipment Hardware Vendor
Author Lance LingerfeltPosted on

Installation of Exchange Server 2019 on Windows Server 2019

I have realized recently that I am an Exchange Messaging Professional, but yet, I have not posted the methodology of how I install an Exchange Server Mailbox Role. So here it is!

Install Windows Server 2019

Exchange Server 2019 requires Windows Server 2019 to run. For my environment, I haven’t necessarily need to follow all the enterprise level design aspects of database numbers to mailbox size ratios, number of servers, front/back end configurations, DAG Implementation, etc… If you want or need to delve into that realm, you can go here. I have need for a single server with only a few databases for a small number of mailboxes, so I am approaching it from that standpoint.

So first, in Hyper-V, I configured my VM with the following specifications:

Processors: 2 procs with 2 cores each – 4 Virtual Processors Total
RAM: 32GB with dynamic memory enabled optional
Drives: 2 .vhdx drives of 120GB each (OS / Exchange Data)
CD: Windows 2019 ISO
Default Settings for the rest of the VM Settings

Next I installed Windows Server 2019 Datacenter with the GUI! You can install it on Server Core if you wish. That information can be found in this link.

I ran through the setup of Windows and installed the OS on my first vhdx drive. I booted up, set the local admin password, and logged in. Once in Windows, I went to the Local Server Settings in Server Manager and configured the following settings:

Set the Date, Time, and Time Zone. (Once in the Domain, this would sync through Group Policy)
Set IE ESC to allow Administrators to have full IE access.
Set Remote Desktop Settings to gain RDP access. (This would be locked down with Group Policy as well once on the Domain)
Set the IP Settings to Static Settings. (DNS Servers, Gateway, WINS, etc…)
Join the server to the Active Directory Domain.
Reboot the VM Server.
Logon to your Domain.
Configure Windows Update Settings. (I have WSUS through Group Policy, this was configured automatically upon reboot)
Download and install all Windows Updates for the server. Then Reboot.
Open Disk Management and configure the secondary vhdx drive to be your Exchange Data Drive.
I configured the drive to be a mounted folder ‘C:\Exchange\Data’ rather than another drive letter as that seems to be the more accepted form of installation for the data drive these days. That is based on the multiple configurations that I have seen for Exchange through experience in Enterprise environments. Again, to each is own and depending on you design specifications, you might want to do that differently.

Next, we need to install the prerequisites for the Exchange Mailbox Server. I have always used practical365.com to get the PowerShell script to install the prerequisites, but couldn’t find the article this time. Great site though! Instead, I got the information and ran the following from an elevated PowerShell Session locally on the server:

As part of the prerequisites you will need to install the following packages onto the server as well:

UCMA Runtime Install
Visual C++ Redistributable Packages for Visual Studio 2013

Once completed, you can begin the install of Exchange. If this is your first Exchange 2019 Server in your Organization, then you will need to run the following to update the Forest, Schema, and Domain so that Exchange will install properly:

NOTE: If you run into Prerequisite issues with the installation due to a “pending reboot”, check out my blog post for information on remediation of that issue.

Now that the environment is prepared for Exchange, you can actually begin the installation. I wanted to make my default database and logs folder to be on the Exchange Data volume that I created, so I included those settings in the setup command. Please look at the reference to the setup.exe switches for more information on that. Here is the command:

Setup should go through the installation via the PowerShell window and complete successfully. Reboot the Exchange Server, then you can then logon to the Exchange Admin Center and begin the process of configuration of how you need to integrate the Mailbox Server into your Server Farm. That configuration is for a later post.

PLEASE CHECK BACK FOR UPDATES!
PLEASE COMMENT!

References:
UCMA Runtime Install
Visual C++ Redistributable Packages for Visual Studio 2013
Install Exchange Server 2019 on Windows Server 2019 Core
Exchange Server Design Planning
Use unattended mode in Exchange Setup
Practical365 on Exchange 2019

Author Lance LingerfeltPosted on

How to transfer FSMO Roles using PowerShell

A rare weekend post for me! HA! I am currently migrating my server environment from VMWare 6.7 to Server 2019 Hyper-V. I have a separate standalone box that I use for my VM backups and as a tertiary DC. Since I had to shut down my VMs in order to convert them, I needed to quickly move my FSMO roles from the DC Virtual Machine to the Standalone box so things would stay running.

I found this great article on how to do that quickly through PowerShell since it is a pain to go into ADUC, ADDT, and setup an MMC for the Schema snap-in.

When you create a domain, all FSMO roles assigned to the first domain controller in the forest by default. You can transfer FSMO roles from one DC to another both the Active Directory graphics snap-ins and the PowerShell command line. Moving FSMO roles using AD PowerShell has the following benefits:

  • You do not need to connect with a MMC snap-ins to the future role owner;
  • Transferring or seizing FSMO roles does not require a connection to the current or future role owner. You can run AD-PowerShell module cmdlets on a Windows Client or Server running RSAT Tools;
  • To seize the FSMO role (if the current owner is not available), it suffices to use an additional parameter -force.

Import the Active Directory Module Into PowerShell:

To get the current forest level FSMO role owners (Domain Naming Master and Schema Master roles) you can use the following PowerShell cmdlet:

To view domain-wide FSMO roles (Infrastructure Master, PDC Emulator and Relative Identifier Master roles):

Transfer FSMO Roles using PowerShell

To transfer FSMO roles between Active Directory domain controllers, we use the PowerShell cmdlet:
Move-ADDirectoryServerOperationMasterRole

To use the Move-ADDirectoryServerOperationMasterRole cmdlet, you must meet the following requirements:

  • There must be at least one DC with a version of Windows Server 2008 R2 or higher
  • PowerShell version 3.0 or newer
  • Active Directory module (2.0  or newer)

NOTE: Unlike the Ntdsutil.exe utility, the Move-ADDirectoryServerOperationMasteRole cmdlet can be performed from any domain computer to migrate the Operations Master roles if you have the appropriate rights (Domain admins and Enterprise Admins).

Import the AD Module:

I needed to move all the roles from one server to the other, so, I ran the following to do so:

NOTE: To simplify the command, you can replace the names of roles with numbers from 0 to 4. The correspondence of names and numbers is given in the table:

PDCEmulator0
RIDMaster1
InfrastructureMaster2
SchemaMaster3
DomainNamingMaster4

So, by having knowledge of these numbers, you can simplify your cmdlet:

NOTE: In the event that the current owner of one or all of the FSMO roles fails, the forced transfer of FSMO roles is performed by the same command, but with the -Force option. Also, after the FSMO roles have been seized, the domain controller from which the roles was seized should never be connected to the domain. You will need to preform a metadata cleanup of the Schema before even thinking about putting that failed server back into production.

Once completed, I ran the previous cmdlets of Get-ADForest and Get-ADDomain to verify that the FSMO roles moved to the destination server.

As of now, my conversion to Hyper-V is going smoothly, although it takes quite a bit of time to convert the hard disks. Thanks again!

HAPPY TROUBLESHOOTING! KEEP SCRIPTING!
PLEASE COMMENT!

Reference:
How To Transfer FSMO Roles Using PowerShell

Author Lance LingerfeltPosted on

Generate Automated E-Mail From Performance Counter Alert

Recently, I was trying to load this IT Blog and noticed that the page took a very long time to load properly. So, first thing I did was logon to my Web Server to check things and saw that the Performance was very slow, almost to a halt. When I was able to pull up Server Manager, I saw the following:

The Performance Alert was pegged at 99+ alerts. I went into the alerts for the performance and saw the following:

My CPU Usage was pegged at 100% for many hours. As I was having issues with the server and could not dig any deeper to find what was running to cause the error other the the w3wp process, I decided to reboot the server.
Rebooting did resolve the issue at hand, but I needed a better way to alert myself when the processor pegged at 95% or more over at least a steady two minute period. That way I could keep the websites going and restart the server if necessary due to an issue with a process pegging the CPU.

I don’t have SCCM or any robust alerting software like we have a my employer, but not to worry, Windows Server has a built in method to generate a task using Task Scheduler when the Performance Counter is above threshold for the time period that you set.

Step 1 – Create your PowerShell Script

The first thing that you need to do is write a Powershell script that when run can send an email. While researching this I discovered many ways to accomplish this task, feel free though to experiment and use what is right for your environment.

Here is an example of a Powershell script that when used in conjunction with Task Scheduler and Perfmon can send an email alert automatically when any user defined performance counter threshold condition is met. In my environment I set this to C:\PowerShell\AlertEMail.ps1

If you notice, this Powershell script takes four arguments and assigns them to variables used in the output. It also saves the computer name to a variable which is also used as part of the output. By doing this the script can be used to send an email on any Perfmon Alerting Counter and on any server without additional customization of the script.

Step 2 – Setup a Scheduled Task To Be Run

In Task Scheduler we are going to Create a new Task as show in the following screen shots.

Create Task

Give the Task a name, you will need to remember it for the next step. Make sure the task is run with an administrative account and is run whether the user is logged on or not.

There are no Triggers to setup here. This Task will actually be triggered through the Perfmon Counter Alert we will set up in Step 3.

On the Action tab you want to define a new action. The action will be to Start a Program and use the following inputs, please adjust for your specific environment.

Program/Script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Add Arguments: -File C:\PowerShell\AlertEMail.ps1 $(Arg0)

Next, set your Conditions and Settings Tabs to the following settings as I did for this purpose. If need be, you might need to set them differently, but these settings should work just fine.

Step 3 – Create the Alert Trigger in Performance Monitor

Open Performance Monitor and Create a new Data Collector Set under Data Collector Sets > User Defined

Name the Alert (This one is for the Processor Usage)
Assign it as a Performance Counter Alert

I needed to monitor the Total % Processor Time for all processors on the server combined, so I set the counter to monitor from the available counters.

I then set the threshold to Alert when the processor is over 95% with Processor Time. I also let it run as the System account, and then saved the settings:

Once you have created the Data Collector Set go into the Properties of it and make sure the Alerting threshold and Sample Interval is set properly for each Performance Counter. Keep in mind, if you sample every 10 seconds then you should expect to receive an email every 10 seconds as long as the performance counter exceeds the threshold you set. I set my Sample Interval to 2 minutes so that if the percentage stays above 95% over that long of a time interval, I should take a look at the server and see what process is pegging the CPU:

If you select the Log an entry in the application event log don’t expect to see any entries in the normal Application event log. It will be written to the Microsoft-Windows-Diagnosis-PLA/Operational log in the Application and Services log directory.

Finally, we have to set an Alert Task that will trigger the Scheduled Task (AlertEMail) that we created in Step 2. You see that we also pass some of the Task arguments which are used by the PowerShell script to customize the email with the exact error condition associated with the Alert:

Once the Data Collector is configured properly you will want to start it.

If you configured everything correctly you should start receiving emails any time that alert threshold is surpassed. I also manually ran the script once from PowerShell with no Argument Data just to make sure the script would generate the email to my E-Mail Address.

Step 4 – Set the Data Collector Set to run upon server startup.

You now have one more step. Whenever you reboot a server the Perfmon Counter Alert will not start automatically. In order to survive a reboot you must run the following at a command prompt.

Note: “Processor Usage Alert” referenced in the script below is the name of my user defined Data Collector Set.

I now have a simple way to monitor my Web Server should this event happen again with the processor being pegged over a two minute period.

PLEASE COMMENT! I LOVE IMPROVEMENTS TO MY SCRIPTS! HAPPY TROUBLESHOOTING!

Reference:
Setup E-Mail Generation from a Performance Counter Alert

Author Lance LingerfeltPosted on

Check Windows Updates Installed via PowerShell

I had an issue last night where a server lost Secure Channel Connection to the PDC Emulator (NETLOGON Event IDs 5719 and 5783). All tests to test the secure channel via PowerShell were failing. (i.e. nltest or Test-ComputerSecureChannel cmdlets) The server essentially needed to be rebooted. I had a dumb dumb in my brain and forgot to check to see if there were any pending Windows Updates, because those need to be installed at the proper time and to a schedule. So, when I ran the following command to reboot:

The Windows Updates were installed inadvertently which could have caused even more issues if they were NOT approved or caused another failure on the server. TO NOT DO THIS IN THE FUTURE, remember to run the following command to shut off the Windows Update Service BEFORE initiating the reboot of the server:

But, the deed was done. NOW, I had to find out quickly what updates WERE installed via PowerShell so that I could alert the proper folks and give them a heads up on possible issues. Luckily, the server did NOT have any issues and the initial problem with NETLOGON was resolved. Here is the command I ran to find out the installed hotfixes filtered by today’s date:

Here was the Output:

Caption=http://support.microsoft.com/?kbid=4480960 
CSName=DC01
Description=Security Update 
FixComments= 
HotFixID=KB4480960 
InstallDate= 
InstalledBy=NT AUTHORITY\SYSTEM 
InstalledOn=2/26/2019 
Name= 
ServicePackInEffect= 
Status= 

Caption=http://support.microsoft.com/?kbid=4480965 
CSName=DC01 
Description=Security Update 
FixComments= 
HotFixID=KB4480965 
InstallDate= 
InstalledBy=NT AUTHORITY\SYSTEM 
InstalledOn=2/26/2019 
Name= 
ServicePackInEffect= 
Status= 

Since there were no issues, I was able to resolve the incident. I did notify the account team though of the inadvertent installation so that they could revert the changes if necessary.

Remember, troubleshooting to resolution is a methodical process, and when in an enterprise environment, you MUST be aware of all factors of change process, even when the resolution is a simple reboot of the affected server.

HAPPY TROUBLESHOOTING!
I LOVE COMMENTS! THANKS FOR READING!

References:
Methods of generating installed updates via PowerShell
Check Windows Update History via PowerShell
Disable or Bypass Windows Update Installation During Reboot/Shutdown of a Server

Author Lance LingerfeltPosted on

Website Availability Checker Script

We get alerts from time to time that SharePoint and Skype Websites might not be loading correctly or not at all. The following is a script that will generate a report, with time and date, showing the availability of the websites in question and the time it takes to load them.

First off, you want to create a text file called URLList.txt that will have the list of websites you want to check. I save the file in the C:\PowerShell directory but you can change the path to where you need it. If this happens often, having this list prepared will help you evaluate the environment quickly and allow you to troubleshoot if necessary efficiently.

URL List Text File

Here is the script:

Here is your output:

PowerShell Output
The report shown in your web browser

HAPPY TROUBLESHOOTING!
MAKE THIS SITE BETTER!
PLEASE COMMENT!

Author Lance LingerfeltPosted on

Moving mailboxes to O365 via PowerShell in Hybrid Configuration

As many of you have knowledge, I am studying for my MS-202 Exam. And, part of the knowledge needed is to be able to migrate mailboxes between on premises and Exchange Online through PowerShell. Here are the steps for the scenario to move a mailbox from on premises to O365:

1. Connect to Exchange Online via PowerShell

If you have read my previous post: Connect to All PowerShell Modules in O365 with one script
You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.

2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.

3. Move a single mailbox.

In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved.
Now we initiate the move with the cmdlet. Similar to what you would do in the GUI, this simple mailbox move cmdlet initiates the move request. It has most of the same parameters as a local move request including BadItemLimit, LargeItemLimit, AcceptLargeDataLoss, etc…
 
Use the following LINK for documentation on the New-MoveRequest cmdlet.

Now with all migration projects, we expect to have to move multiple mailboxes in a single batch. The following will show the process for moving mailboxes in bulk from on premises to O365:

1. Connect to Exchange Online via PowerShell

If you have read my previous post: Connect to All PowerShell Modules in O365 with one script
You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.

2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.

3. Move multiple mailboxes in a single batch.

In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved.

This time you want to create a CSV file using the alias or emailaddress as your header and then list the appropriate value for all the users in your batch group. Save the file locally as MigrationBatch01.csv or a name of your choice.

Use EMailAddress
 OR
 Alias as the header

Next you initiate the mailbox moves. When specifying the mailbox identity in the cmdlet, use the respective header in your variable declaration (either $user.EMailAddress OR $user.Alias)

Use the following LINK for documentation on the New-MoveRequest cmdlet.

GOOD LUCK WITH YOUR MIGRATIONS!
HAPPY TROUBLESHOOTING!

References:
Moving Individual Mailboxes to O365
Move Mailboxes in Bulk to O365
PowerShell Mailbox Migration to O365
Connect to all PowerShell Modules in O365 with one script
New-MoveRequest Microsoft Document

Author Lance LingerfeltPosted on

Exchange DAG Replication Problem: An established connection was aborted by the software in your host machine

I had an issue with a four node DAG where the DR site with two of the DAG members were having replication issues. It was only technically affecting one DAG Member though. The copy queue length was really high and the logs were not committing to the database. A Test-ReplicationHealth cmdlet test told that the copy queue length for the affected database copy was high. No other databases were affected as there were eight databases on this DAG Node. The issue was that the log files were not replicating properly to the one DAG member for that database, causing the log file drives on all the other DAG members to build and become full:

EX04 DAG member has high Copy Queue Length
The purple member (EX04) free space is different from the other three DAG members

Circular Logging was turned on, but since the db was NOT in sync, the logs could NOT truncate properly which rendered CL useless. What was being done to stave the issue was to suspend the database copy of the affected DAG member (EX04), then resume the copy. The logs would replay and commit to the database copy on the DAG member, but over a short period of time, the same issue would arise again, as shown in this graph:

You can see the other DAG members start dropping in free space

There were absolutely no errors in the Event Viewer showing this replication issue. After some research, I ran the following cmdlet showing a particular output parameter that gave me the actual problem:

Get-MailboxDatabaseCopyStatus DAG1DB01 | ft -a -wr Name, Status, IncomingLogCopyingNetwork

Output with the actual error listed for the DR DAG members.

The operative error here was: {An error occurred while communicating with server ‘EX01’. Error: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.} 

Now even though only EX04 was actually having problems with its log replication, both DR members EX03 & EX04 were having the same problem. Again, there were NO events in event viewer showing this issue. I next did some connectivity tests to EX01 from EX04 even though the error said there was an established connection that was broken.

Ping EX01 -f -l 1472

Now the -f states do NOT fragment the packet and send it as a whole to the destination.
The -l states the packet/buffer size you want sent. In this case 1472 bits.
By doing this, you are able to assure that a router or switch is NOT segmenting the packets, packet segmentation of replication logs can cause data corruption and replication issues.

That test passed successfully. I also did a trace route to assure there was no packet loss on the route to the replicating server. That test passed successfully.

I next checked the DAG Network to assure that all networks were working for replication. Now, in this scenario, there was only ONE DAG Network, there was NOT a separate Replication Network. I did not design the DAG and limitations most likely came into play during the design. From my experience, you setup a separate replication network for replication only, but if your network has enough bandwidth, and the design calls for simplification, you can use one DAG network in your design.

Get-DatabaseAvailabilityGroupNetwork | fl 

RunspaceId : a1600003-8074-4000-9150-c7800000207f 
Name : MapiDagNetwork 
Description : 
Subnets : {{192.168.1.0/24,Up}, {192.168.2.0/24,Up}} 
Interfaces : {{EX01,Up,192.168.1.25}, {EX02,Up,192.168.1.26},{EX03,Up,192.168.2.25}, {EX04,Up,192.168.2.26}} 
MapiAccessEnabled : True 
ReplicationEnabled : True 
IgnoreNetwork : False 
Identity : DAG1\MapiDagNetwork 
IsValid : True 
ObjectState : New 

All the DAG Network Members were up and not showing errors. I next did a telnet session to EX01 over the default DAG replication port 64327 to see if there would be any connectivity issues to EX01:

telnet EX01 64327

That test was successful and there were no connectivity issues to EX01 from EX04. Again, there was only ONE database out of eight that was having replication problems. After mulling over the problem, it was decided to restart the MSExchangeRepl service on EX03 AND EX04 since the error was present on both DAG members. We would then, suspend the database copy and resume the database copy on the affected servers.

Run on EX03:
Restart-Service MSExchangeRepl
Suspend-MailboxDatabaseCopy DAG1DB01/EX03 -Confirm:$False
Resume-MailboxDatabaseCopy DAG1DB01/EX03 -Confirm:$False

Run on EX04:
Restart-Service MSExchangeRepl
Suspend-MailboxDatabaseCopy DAG1DB01/EX04 -Confirm:$False
Resume-MailboxDatabaseCopy DAG1DB01/EX04 -Confirm:$False

After monitoring the databases and log drives, the issue was resolved and replication started functioning properly.

Log Drive Available Space Returned to Normal for DAG members

PLEASE COMMENT! I WELCOME SUGGESTIONS, TIPS, ALTERNATIVE TROUBLESHOOTING! HAVE A GREAT DAY!

Author Lance LingerfeltPosted on

Checking Drive Space Volumes for DAG DB members through PowerShell

I had received a weird alert for a DB volume for a DAG member being below threshold. This was odd to me due to the fact that there were four DAG members and we only received an alert for one. I went into Azure Log Analytics and ran the following query to render a graph for the past 14 days showing the percent free space of the volume for all the DAG members.

Thanks Georges Moua for the query script!

Now the reason I can run the query this way is due to the fact that the Design of the DAG was correctly done and the DB folders are identical on all DAG members. The query rendered the following chart:

As you can see the Green DAG member is way below the other DAG members.

I next went to an Exchange Server in the DAG and got the volume data for all the members in the DAG:

EX02’s volume free space is far below the other DAG members

I went on EX02 and found that there was a subfolder named “Restore” that was not present on the other servers. I ran the following script to get the size of that folder in GB:

The folder size was 185 GB. Removing that folder, along with all subfolders/files, would balance the free space to the other DAG members. I ran the following cmdlet to remove the folder and all subfolders/files:

This remediated the alert and balanced the drive space across all DAG members.

POST YOUR COMMENTS OR QUESTIONS!
HAPPY TROUBLESHOOTING!

Author Lance LingerfeltPosted on

What the Hybrid Configuration Wizard Performs in the background and configuring Hybrid Co-Existence with Exchange Online

I’m working on getting certified in Exchange Hybrid Scenarios and Exchange Online configuration as part of my skill set for Exchange. In doing so, I had successfully implemented a complete Full Hybrid Exchange Environment between my Exchange Online Tenant and my On Premises Exchange 2019 Environment last evening.

I wanted to give an update that was posted to my LinkedIn Posting on this. Thank you Brian Day for the vote of confidence and caution that running these cmdlets manually is not supported by Microsoft and that the HCW, like all the Online Microsoft Products, is constantly changing and being updated.

Important Note

As preparation, I bought some Exchange Online Plan 1 licenses which give me a 50 GB mailbox limit and basic mailbox functionality. It does not include the more advanced features such as ATP, or DLP. I am running most of those features through my On Premises Environment. I mainly wanted to be able to place mailboxes in the cloud and have a hybrid setup. My plan was to have mail flow continue through my On Premises environment so that my Exchange Server features would be used and I would not have to change any MX or SPF records. I also had my certificates in place for SSL and OWA so I would want keep mail flow routed that way, through on premises. I do want to be able to have Free/Busy lookups cross-premise so federation would have to be enabled as well. I would also have to enable the MRS proxy on my Exchange Server so that mailbox migration could be implemented cross-premise. I also have previously configured Azure AD Sync along with ADFS for Single Sign On. In my case, another server was not needed as I didn’t have enough mailboxes or real need to split my frontend and backend deployment. Running the Hybrid Configuration Wizard would not open any new ports or change any existing port traffic that was already configured on my firewall. These are just a few of the considerations that need to be looked at when considering a hybrid integration.

Here is a great article to read for the prerequisites
Exchange Hybrid Deployment Pre-requisites

So, once I had all those considerations handled in my design, I ran the Hybrid Configuration Wizard. What I want to do in this blog post is to go through the steps that the wizard does in the background to setup the Hybrid Environment as you go through the Wizard.

I mainly used the following blog post as a reference, but have approached it differently by diving into the cmdlets that are run during the process:

Exchange/Office 365 Hybrid Configuration Wizard – step by step guide

1. The HCW validates the On-premises and Online Exchange Connection.

The Hybrid Configuration Wizard checks if it is possible to connect to both servers with PowerShell. It runs the Get-ExchangeServer cmdlet on premises after resolving the server in DNS. It then connects to Exchange Online, authorizing the connection:

Authority=https://login.windows.net/common Resource=https://outlook.office365.com ClientId=abcdefgh-a123-4566-9abc-2bdflancelin

2. The HCW collects data about Exchange configuration from the on-premises Active Directory

The Wizard gathers information about the local domain. In order to do that, the HCW executes a series of cmdlets.

These include, in order:

3. The HCW collects information on the Exchange online (Office 365) configuration

This task repeats what has been done in the previous step, only for the Exchange online, instead of the on-premises one.

The cmdlets include, in order:

4. Federation Trust is determined. If not present, a new Federation Trust and the required certificate will be created on the local Exchange Server

You will be prompted in the Wizard to create a Federation Trust if not present. The following articles explain Federation and its requirements:

Understanding Federation – Link Here
Understanding Federated Delegation – Link Here
Create a Federation Trust – Link Here

If the activity is finished successfully, a new certificate should appear on the on-premises Exchange Certificates list. The new certificate includes “Federation” in its Subject field. To make sure the certificate is there, you can run a cmdlet: Get-ExchangeCertificate | ft -a -wr


The results will look like this

5. The HCW creates a new Hybrid Configuration Object in the local Active Directory

The HCW will run cmdlets based on the information you provide in the HCW for the certificate, the on premises Exchange Server, the domain(s), and what features you want turned on:

It then checks the settings through the following cmdlets:

It then enables Organization Customization for both environments through this cmdlet:

6. Configuration is then completed to modify the settings on the on premises Exchange environment 

EmailAddressPolicy – HCW adds address @tenant.mail.onmicrosoft.com
The HCW configures remote domains – adds tenant.mail.onmicrosoft.com and tenant.onmicrosoft.com
The HCW adds a new accepted domain – adds tenant.mail.onmicrosoft.com

Some of the cmdlets run:

7. The HCW Configures the Organization Relationship between the local server and the cloud.

This configuration is not necessary in minimal hybrid deployment. Since I have a full hybrid deployment configured, the cmdlets were run as needed to configure it. Thanks to the correct configuration, it is possible to synchronize free/busy status of mailboxes and their elements between the on-premises Exchange Environment and Exchange online. 

Some of the cmdlets run in the process:

8. The HCW and setting connectors on both Exchange servers
The HCW checks to see if the connectors are there, if not, it sets them up. During this workflow, four connectors are set – one receive and one send connector for each server. Those connectors guarantee the mail flow between the on-premises and Exchange Online.

Some of the cmdlets run in the process:

The Intra-Organization is set as well:

9. The HCW configures OAuth Authentication across the Hybrid

This LINK explains how OAuth is configured between Exchange On Premises and Exchange Online. It’s a very good article to read as it shows how to get the Modern Authentication style working. Now the HCW does this for you and at the end of the article, you can run cmdlets to test the validity of the configuration.

If you want to go into a deep dive about how the Hybrid Authentication works, see the following:
Deep Dive Into Hybrid Authentication – from the MS Exchange Team Blog

Here are some of cmdlets run during this process workflow:

Again, look at both of those links to get a little more detail as to what each cmdlet does and how it sets up OAuth. Here are the two cmdlets used to test OAuth:

10. Enable MRS Proxy for Migration

In order to be able to move mailboxes between Exchange On Premises and Exchange Online, you have to enable the Exchange Web Services Virtual Directory to use the MRSProxy (Microsoft Replication Service proxy). You also have to set your EWS Virtual Directory to use Basic Authentication. You’ll want to do this before running the HCW or else you will receive the following error when the HCW validates the Migration setup and configuration:

Microsoft.Exchange.Migration.MigrationServerConnectionFailedException: The connection to the server ‘mail.ldlnet.net’ could not be completed. —> Microsoft.Exchange.MailboxReplicationService.RemoteTransientException: The call to ‘https://mail.ldlnet.net/EWS/mrsproxy.svc’ failed. Error details: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. –> The remote server returned an error: (403) Forbidden.. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. —> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The remote server returned an error: (403) Forbidden.

Some of the cmdlets run to test Migration and MRS Proxy Settings are as follows:

11. Final HCW Configuration and cleanup.

The HCW runs from final cmdlets to finish up the installation of the Hybrid environment. Here are the cmdlets run:

All this information was found in the setup logs that are in the following directory
C:\Users\%username%\AppData\Roaming\Microsoft\Exchange Hybrid Configuration

REFERENCES
Understanding Federation
Understanding Federated Delegation
Create a Federation Trust
Hybrid deployment prerequisites
Exchange Specific OAuth 2.0 Protocol Specification
Understanding WS-Security
JSON Web Tokens
Using OAuth2 to access Calendar, Contact and Mail API in Office 365 Exchange Online
Configurable token lifetimes in Azure Active Directory (Public Preview)
OAuth Troubleshooting
Principles of Token Validation
Troubleshooting free/busy issues in Exchange hybrid environment
How to configure Exchange Server on-premises to use Hybrid Modern Authentication
Microsoft 365 Messaging Administrator Certification Transition (beta)
Microsoft 365 certification exams
Exchange Server build numbers and release dates

PLEASE LEAVE QUESTIONS, COMMENTS, UPDATES! I WOULD LOVE TO HEAR FROM YOU!

Author Lance LingerfeltPosted on

The Complete Guide To PowerShell Punctuation

In this short little snippet, I wanted to provide a good download that I had found. In my searching for more knowledge through PowerShell, I found this great PDF file that has listed all the different symbols used in PowerShell Scripting and their meanings.

Here is the download:

PowerShell Punctuation Wall ChartDownload
Example of the available file.

Please feel free to download it and use it in your education of PowerShell. I have knowledge that I will be using it. The website I got it from has some good explanation of functions and punctuation for PowerShell as well. See below…

Reference:
PowerShell Punctuation

Author Lance LingerfeltPosted on

Measuring CPU Processor Times Per Core Across Multiple Servers through PowerShell.

I want to thank Jason Field for the bulk of this script!

Our team was presented with an issue where we needed to measure the CPU Percentage Processor Times for Each Core within the Physical Processor and be able to output that data quickly through PowerShell. We all have knowledge that Performance Monitor can do this through a GUI, but it is very difficult to be able to output that data to a file in a manner that can be easily read. We had an original PowerShell cmdlet that would accomplish this for the total percent processor time for all cores over a one minute period:

Our challenge was to be able to do this per core, over the same time period, and get the average for each core, so that we could measure the output accordingly. I had been working on a script that was able to run the command, but not in a parallel fashion. The script was running in sequence and was taking way longer than one minute to complete. Frustrating to say the least.

In comes Jason Field, showing me the meaning and value of the back tick as well has how to run and monitor job functions in a PowerShell Script so that the task could be completed, as needed, across a server array.

The main purpose of the (back tick) is that it allows for variables to be used in the script block on the remote server instead of being filled in before creating the script block.

Here is the script:

Sample Output from the Script.

What I learned from this script is that the back tick ” ` “ allows for multiple commands to be run in a sub-routine within the script job and be gathered before the main script command is run and the output given. This gets past the multi-threading issue I was having with my original script. The script can then be run across multiple servers using the Invoke-Command cmdlet or over a remote PowerShell session as a Job. The jobs can then be monitored as the scripts finish across the multiple servers in the time period given for the samples. I had modified the script to do the multiple samples and then take the average of the CookedValue per the original cmdlet. I could not, however, get the ExpandProperty parameter to work with the script.

Please, as I am still learning, if you see an error with this script, please alert me with a comment or contact me directly so that I can update the script properly. 

Thanks again Jason! It really was like a light bulb going “DING” when I figured it out with concern to the back tick. I had also been having issues with the active job monitoring process. It was a real help!

PLEASE COMMENT, SHARE, AND HAPPY TROUBLESHOOTING!

Author Lance LingerfeltPosted on

PowerDNS Script

I was compiling some scripts to be able to modify DNS records in my previous post. While browsing through different scripts in the TechNet Gallery, I came across the following Script that provides a menu, options, and different settings which really make it a great script to use if you do a lot of DNS Modification and want to do it through PowerShell.

Here is the link to the original script page, but I have updated and modified the script to include being able to add/remove DNS Zones as well.

The DNS Zone functions have not been tested as of yet. I still have to get on my server farm at home and run this. It will save time though with me having to switch servers when adding a bulk list of DNS zones for my website farm. Play with the script and let me know what you think!

Author Lance LingerfeltPosted on

Removing a DNS Record through Powershell

In most environments, an admin usually just jumps on the server that they need to work from and does their work from there. An example of this would be an admin working on an IIS Web server and needing to remove a DNS A record from DNS without having to logon to the DNS server itself so that they can quickly make their changes in IIS.

A quick way to do this would be to run the following ps1 script in PowerShell in order to be able to remove the record quickly:

Sample Output from the script.
Sample Output from the Script removing DNS A Record: test.ldlnet.local

Now this works for a single DNS A Record. If there are multiple IPs for the same DNS record, for example, test.ldlnet.local points to both 192.168.1.23 and 192.168.1.24, then you probably need to run the following script listed here to keep the script from failing with an error. I have also expanded the entries to help the input be more specific:

Output from RemoveDNSRecord.ps1 for removing DNS A Record test.ldlnet.local with IP of 192.168.1.24

I have found some other good scripts that I will post to the blog to help manage DNS records through PowerShell. This should get things started for now. Happy Troubleshooting!

Author Lance LingerfeltPosted on

MaxConcurrentAPI Script for Netlogon Issues

I get incidents from time to time that deal with Netlogon Service Issues. For example: Semaphore Waiters, Semaphore Timeouts, Semaphore Acquires, etc…

Here is a script I got from the Microsoft Gallery
In some enterprise environments the sheer volume of NTLM authentication can produce performance bottlenecks on servers. To help make the problem easier to detect, this PowerShell script was written.