The Windows Time Service, Hyper-V Hosts, and DCs that are VMs.

The sheer craziness of it all! I noticed that my clocks were off on my servers by FOUR minutes. I had originally set in group policy for the PDC emulator for my domain, a VM on one of my Hyper-V hosts, to get the time from the Public NTP hosts. I then configured a group policy to have all the other machines get their time from the PDC Emulator.

This was working great for me until I realized that my Hyper-V hosts were actually controlling the time of the VMs. They were also configured to get the time from the PDC Emulator, but essentially, due to how Hyper-V is configured, the PDC Emulator VM was getting the time from the Host. So, once the time got thrown off, everything went wacky on me!

I’d read through a couple of articles and found the configuration flaw of Hyper-V and the need for those servers to get their time from the external NTP hosts as well as be configured as NTP servers themselves. This totally went against my Group Policy configuration which caused the issue!

Luckily, I had a stand alone server that is a tertiary DC in the domain not running Hyper-V. I was able to get my time synced again properly after performing the following configuration.

  • I had to move the FSMO roles to the tertiary DC with the following cmdlet:

  • I then made sure the tertiary DC was syncing time correctly by running the following on that server:

  • I then removed the Group Policy Object for syncing the time source to the DC that I had linked to my Hyper-V Servers OU in Active Directory
  • Ran a gpupdate /force on the Hyper-V host to remove the policy there
  • I then had to reconfigure the Hyper-V hosts to be NTP Servers and clients that got their time from a public NTP server:

The one problem Hyper-V host that was syncing with the DC VM would not change settings via Group Policy nor through the w32tm cmdlet. I even went into the registry and tried to modify the following keys to make the changes stick:

The values would just not change, most likely due to the time not being synchronized. I had to reboot the server and then run through the process again in order for the changes to stick.

I did look at another article that said to do the following on the DC VM in order for time NOT to sync with the Hyper-V Host:

Go into Hyper-V console on the host machine, right-click on the client VM AD server, and select Settings. Once in here, on the left look under:

Management –> Integration Services
Untick Time Synchronization
Click Apply/OK

Virtual Machine Settings within Hyper-V

Things are running smoothly now. Please view the references at the bottom of the post. There are a couple of great articles about the Time Synchronization process with Hyper-V and why it needs to be setup the way I have it now. I wished I had read it before I originally set this up. I will post the article about getting group policy to handle the time sync process. Just remember, if your PDC Emulator is a VM, don’t sync it to a public NTP server. Sync it to your Hyper-V Host and have the Host sync publicly.
In the long run, I think it is a good design solution to have your Hyper-V hosts time synced to the Public NTP servers than having to remember to configure each VM DC you create to NOT time sync with the host. To each is own though, and one thing I learned from working Microsoft, there are multiple ways to get to the same goal that are technically sound methods.

THANKS FOR READING!
PLEASE COMMENT!

REFERENCES:
Setup of NTP on Hyper-V servers
Time Synchronization in Hyper-V
“It’s Simple!” – Time Configuration in Active Directory
NTP Circular Time Sync – Windows Server 2012 R2 / Hyper-V

Error 801c0003 when joining computer to Azure AD

I just received my new laptop for my current project and was setting up Windows 10 to join the company Azure AD domain. When I got to the part where you join, I received the following error:

Error Joining Computer to Azure AD

Turns out that my account is unable to domain join a device to the tenant. This is easily solved though. You have your tenant admin perform the following:

Go to Azure Active Directory -> Devices
Check the device settings, in particular the options:

Users may join devices
Maximal number of devices

Azure AD Settings Page

Now, in my case, I did not have access as I am NOT a tenant admin:

So, I am currently waiting for my IT department to resolve the access issue and grant me access to join the device to the domain. Just be sure to look at this if you’re having issues setting up your Windows 10 device to join your Azure tenant!

HAPPY TROUBLESHOOTING!
POSITIVE ENERGY!

References:
Issue Joining A Device To An Azure AD Tenant Domain

How to transfer FSMO Roles using PowerShell

A rare weekend post for me! HA! I am currently migrating my server environment from VMWare 6.7 to Server 2019 Hyper-V. I have a separate standalone box that I use for my VM backups and as a tertiary DC. Since I had to shut down my VMs in order to convert them, I needed to quickly move my FSMO roles from the DC Virtual Machine to the Standalone box so things would stay running.

I found this great article on how to do that quickly through PowerShell since it is a pain to go into ADUC, ADDT, and setup an MMC for the Schema snap-in.

When you create a domain, all FSMO roles assigned to the first domain controller in the forest by default. You can transfer FSMO roles from one DC to another both the Active Directory graphics snap-ins and the PowerShell command line. Moving FSMO roles using AD PowerShell has the following benefits:

  • You do not need to connect with a MMC snap-ins to the future role owner;
  • Transferring or seizing FSMO roles does not require a connection to the current or future role owner. You can run AD-PowerShell module cmdlets on a Windows Client or Server running RSAT Tools;
  • To seize the FSMO role (if the current owner is not available), it suffices to use an additional parameter -force.

Import the Active Directory Module Into PowerShell:

To get the current forest level FSMO role owners (Domain Naming Master and Schema Master roles) you can use the following PowerShell cmdlet:

To view domain-wide FSMO roles (Infrastructure Master, PDC Emulator and Relative Identifier Master roles):

Transfer FSMO Roles using PowerShell

To transfer FSMO roles between Active Directory domain controllers, we use the PowerShell cmdlet:
Move-ADDirectoryServerOperationMasterRole

To use the Move-ADDirectoryServerOperationMasterRole cmdlet, you must meet the following requirements:

  • There must be at least one DC with a version of Windows Server 2008 R2 or higher
  • PowerShell version 3.0 or newer
  • Active Directory module (2.0  or newer)

NOTE: Unlike the Ntdsutil.exe utility, the Move-ADDirectoryServerOperationMasteRole cmdlet can be performed from any domain computer to migrate the Operations Master roles if you have the appropriate rights (Domain admins and Enterprise Admins).

Import the AD Module:

I needed to move all the roles from one server to the other, so, I ran the following to do so:

NOTE: To simplify the command, you can replace the names of roles with numbers from 0 to 4. The correspondence of names and numbers is given in the table:

PDCEmulator0
RIDMaster1
InfrastructureMaster2
SchemaMaster3
DomainNamingMaster4

So, by having knowledge of these numbers, you can simplify your cmdlet:

NOTE: In the event that the current owner of one or all of the FSMO roles fails, the forced transfer of FSMO roles is performed by the same command, but with the -Force option. Also, after the FSMO roles have been seized, the domain controller from which the roles was seized should never be connected to the domain. You will need to preform a metadata cleanup of the Schema before even thinking about putting that failed server back into production.

Once completed, I ran the previous cmdlets of Get-ADForest and Get-ADDomain to verify that the FSMO roles moved to the destination server.

As of now, my conversion to Hyper-V is going smoothly, although it takes quite a bit of time to convert the hard disks. Thanks again!

HAPPY TROUBLESHOOTING! KEEP SCRIPTING!
PLEASE COMMENT!

Reference:
How To Transfer FSMO Roles Using PowerShell

Connect to all PowerShell Modules in O365 with one script

Let’s say you’re an admin that needs to connect to Office365 via PowerShell often. Now, there are many different websites or blogs that will show you how to connect to each session via PowerShell. That can cause a headache since you can end up having five different PowerShell sessions running in five different windows. You end up having to enter a username and password all those times, which can become time consuming.

I want to show you here how to combine all those sessions into one script where, if you’re security is tight enough on your computer, you don’t even have to enter credentials. This way, you can click on one icon and pull up all the O365 PowerShell commands that you’ll need to manage your organization.

First you need to download the following PowerShell Module Installation Files so that your PowerShell Database will have the correct modules installed:

Microsoft Online Service Sign-in Assistant for IT Professionals RTW
Windows Azure Active Directory Module for Windows PowerShell v2
SharePoint Online Management Shell
Skype for Business Online, Windows PowerShell Module

Next, we want to setup the CLI (Command Line Interface) to be too cool for school. I have learned it helps to have knowledge of how to customize the CLI window. You can do all of this in PowerShell ISE or Notepad, which ever you prefer. Here are the commands for the script that I use to setup the CLI:

Next, you want to set your Execution Policy and put in your credentials so that you won’t be prompted to enter the user credentials when you run the script.

NOTE: MAKE SURE YOU KEEP YOUR SCRIPT SAFE AS THE CREDENTIALS ARE VISIBLE WITHIN THE SCRIPT IN PLAIN TEXT!

You can, alternatively, set your script to prompt for credentials every time by using the following:

$LiveCred = Get-Credential

Here is that part of the script:

Now we get into the importing of the modules for each O365 service:

Get the MSOnline Module:

Connect to the MSOnline Service:

Connect to Azure AD PowerShell:

Connect to SharePoint Online PowerShell:
NOTE – MAKE SURE YOU CHANGE TO YOUR COMPANY NAME IN THE URL!!

Connect to Exchange Online PowerShell:

Connect to Skype For Business Online PowerShell:

Connect to the Security & Compliance PowerShell:
NOTE – This one I still get “Access Denied” when trying to connect. I have looked for an answer to that issue, but have not found one. Please comment with a link if you have an answer so that I can update this script!

Lastly, put in a note to show that the PS load is completed:

So Here is the final script in its entirety:

Now you can create your icon for your desktop so that you can easily access the script. I would save the script to your Scripts directory.

That will usually be C:\Users\’username’\Documents\WindowsPowerShell\Scripts or wherever directory you choose.

To start, right click the desktop and choose New > Shortcut
In the Target Field, enter the following for your PowerShell Shortcut, pointing to the path of your script:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -ExecutionPolicy Unrestricted -File “C:\Users\username\Documents\WindowsPowerShell\Scripts\ConnectO365All.ps1”

Click on the Advanced button and check the box: Run As Administrator
Under the General Tab, name your shortcut: (CompanyName) O365 All PowerShell
Click OK to save the shortcut to your desktop.

LAST BUT NOT LEAST, RUN THE FOLLOWING COMMAND BEFORE EXITING OR CLOSING YOUR POWERSHELL WINDOW. THIS WILL REMOVE ALL THE SESSIONS YOU’VE CONNECTED TO:

Get-PSSession | Remove-PSSession

HAPPY SCRIPTING!
LEARN, DO, LIVE!

References:
Connect to all O365 Services in one PowerShell Window
How to connect to all O365 Services through PowerShell
Connecting to Office 365 “Everything” via PowerShell

MaxConcurrentAPI Script for Netlogon Issues

I get incidents from time to time that deal with Netlogon Service Issues. For example: Semaphore Waiters, Semaphore Timeouts, Semaphore Acquires, etc…

Here is a script I got from the Microsoft Gallery
In some enterprise environments the sheer volume of NTLM authentication can produce performance bottlenecks on servers. To help make the problem easier to detect, this PowerShell script was written.

Execution:

Now, I modified this script taking out the clear screen parameter so that I could be run against multiple servers. Place the script in your Scripts directory and name it CheckMaxConcurrentApiScript.ps1

First, in PowerShell, gather your list of servers:

Or

Next, run the command to run the ps1 against those servers:

Or

Sample Output:

DC03
Detection Time : 12/13/2018 7:56:16 PM
Problem Detected : False
Server Name : DC03
Server Role : Domain Controller
Domain Name : ldlnet.org
Operating System : Microsoft Windows Server 2008 R2 Enterprise
Time Since Last Reboot : 4 days 22 hours
Current Effective MaxConcurrentApi Setting : 10
Suggested MaxConcurrentApi Setting (may be same as current) : 10
Current Threads in Use (Semaphore Holders) : 0
Clients Currently Waiting (Semaphore Waiters) : 0
Cumulative Client Timeouts (Semaphore Timeouts) : 17
Cumulative MaxConcurrentApi Thread Uses (Semaphore Acquires) : 3493999
Duration of Calls (Avg Semaphore Hold Time) : 0

EXCH02
Detection Time : 12/13/2018 8:00:53 PM
Problem Detected : False
Server Name : EXCH02
Server Role : Member Server
Domain Name : ldlnet.org
Operating System : Microsoft Windows Server 2008 R2 Standard
Time Since Last Reboot : 4 days 23 hours
Current Effective MaxConcurrentApi Setting : 10
Suggested MaxConcurrentApi Setting (may be same as current) : 10
Current Threads in Use (Semaphore Holders) : 0
Clients Currently Waiting (Semaphore Waiters) : 0
Cumulative Client Timeouts (Semaphore Timeouts) : 570
Cumulative MaxConcurrentApi Thread Uses (Semaphore Acquires) : 1682257
Duration of Calls (Avg Semaphore Hold Time) : 0

Hopefully, this script will assist you with gathering the needed information to help you balance the netlogon load between your servers when needed in your environment.

HAPPY TROUBLESHOOTING!

Protected AD Groups and the problems they can cause accounts

I have run into this issue over the years with accounts being in the Domain Admins group and having issues running PowerShell cmdlets as well as not being able to connect to ActiveSync from a mobile device with the account.

These issues are due to the AdminSDHolder Template in AD and the SDProp Process that is run every 60 Minutes in AD.
This is explained in fantastic detail through the following Microsoft article: Protected Accounts & Groups In Active Directory

Here is an example of an issue that occurred in one of the environments that I was managing. A user was trying to run the following AD cmdlet in PowerShell on DC01:

The user got the following error when the cmdlet was executed:

Set-ADUser : Insufficient access rights to perform the operation
At line:1 char:1
+ Set-ADUser lancel -Server dc01.ldlnet.org -Replace @{title=”Senior O …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo: NotSpecified: (lancel:ADUser) [Set-ADUser], ADException
+ FullyQualifiedErrorId : ActiveDirectoryServer:8344,Microsoft.ActiveDirectory.Management.Commands.SetADUser

The issue was that the admin account used to run the cmdlet was in the Domain Admins group and was not inheriting permissions per the AdminSDHolder template that was applied to the account:

I checked to see that the admin account was in a protected group:

I next went to the Security Tab > Advanced Button and saw that the Enable Inheritance button was visible:

I’ve circled where to look in the window.

This verifies that the account is protected due to being in the Domain Admins group. Now, there are two workarounds for this particular error that we were experiencing.

  1. Click the Enable Inheritance button. This will cause the permissions to be inherited temporarily. When SDProp is cycled again, the account will lose any inherited permissions and will be essentially “broken” again. This is not good if you’re going to be running cmdlets regularly to modify AD Accounts.
  2. The preferred method to work around this issue is to set the -Server parameter to point to a different DC than the one you are on. So, essentially, we tell the cmdlet to execute on DC02 when running the cmdlet from DC01.

Either method will allow the cmdlet to execute successfully and modify the object. You would think that Microsoft would have noticed this issue with running an admin cmdlet for Active Directory, but they have not fixed this issue as of yet nor do i think they plan to. I would just go with workaround number two and remain sane.

Another example of this Protected Group issue comes with an account in a Protected Group that has a mailbox not being able to connect to Exchange ActiveSync when setting up their mobile device.

  • You usually get a 500 error on the device that you cannot connect.
  • You will also see event 1053 in Event Viewer alluding to not having sufficient access to create the container for the user in AD.

Read this page for more information: Exchange ActiveSync Permissions Issue with Protected Groups

So, in your endeavors admins, keep this in mind when running into these types of problems. Happy Troubleshooting!

PowerShell Script to log NETLOGON Events 5719 and 5783, then test the Secure Channel to verify connectivity

In my support role, we would get nightly alerts showing disconnection to the PDC from other DCs and Exchange Servers, giving the following events:

DC02
10/31/2018 23:20:28 5719
NETLOGON
This computer was not able to set up a secure session with a domain controller in domain LDLNET due to the following:
The remote procedure call was cancelled.
This may lead to authentication problems. Make sure that this computer is connected to the network. If the problem persists, please contact your domain administrator.

ADDITIONAL INFO
If this computer is a domain controller for the specified domain, it sets up the secure session to the primary domain controller emulator in the specified domain. Otherwise, this computer sets up the
secure session to any domain controller in the specified domain.

DC03
10/31/2018 23:18:58 5783
NETLOGON
The session setup to the Windows NT or Windows 2000 Domain Controller \\DC01.LDLNET.ORG for the domain LDLNET is not responsive. The current RPC call from Netlogon on \\DC03 to \\DC01.ldlnet.org has been cancelled.

In order to validate the secure channel, you normally run the nltest command (you can also run the Test-ComputerSecureChannel PowerShell cmdlet) to verify the connectivity to the PDC on the secure channel. The scenario is though that multiple DCs or Exchange servers are having multiple events at a similar time due to a network hiccup that brought the secure channel offline between the two Servers.

Our team at the time was getting a lot of alerts generated and it was taking an inordinate amount of time to validate and test. In an effort to provide an efficient solution for this issue, I compiled a PowerShell ps1 script to first validate the events posted in the past three hours, and then secondly, test all the DCs and Exchange Servers for the Secure Channel Connectivity:

NOTE: This script needs to be run on a server that has the Exchange and Active Directory RSAT tools for PowerShell.

I can’t really put out the output since it will have customer PII, but you will see where it will list the DC/Exchange Server Name, show the events, then run the test. You can then troubleshoot from there. Also, know that the secure channel test will FAIL when run on the PDC Emulator DC. The PDC Emulator cannot run a secure channel test on itself.

Please, if you have any questions or comments, please leave some feedback! Happy Troubleshooting!