Create a custom Windows 10 image for distribution using and ISO image.

I’ve been currently assisting with onboarding at my new Full Time Contractor position at Microsoft. All of the new FTCs received laptops and needed to have the newest build of Windows 10 installed. The issue was that all of our laptops came with Windows 10 Professional and we needed to upgrade them to Enterprise edition.

After finding a working key for Enterprise Edition, we were still having issues joining the MS Azure domain so that we could get all of the needed software properly to being onboarding with Microsoft.

So, after going through a couple of re-images of my laptop with some failures attached to that, I finally was able to get the process down so that time would not be waisted for the oncoming new hires once they received their laptop. The issue was getting the correct build of Windows 10 and getting the proper apps installed in an efficient manner. Since the onboarding process was quickly moving, I needed to find a way to help streamline the process so the others would not have to go through all the mess I went through to get everything setup.

So, I began looking for a way to create a customized ISO for the build that would already have apps, settings, and customizations installed. I found this great article that details the process. I wanted to re-post this article here showing the steps I took to create the customized image by creating a VM in Hyper-V and then converting that completed image to an ISO that could be downloaded and utilized for the installation.

Creating a customized ISO image with pre-installed software and no user accounts

  • A generalized ISO image without any pre-set user accounts, with pre-installed software, desktop, File Explorer and Start customizations will be created.
  • All customizations and personalization will automatically be applied to all new user accounts
  • Clean install will perform a normal OOBE, asking for regional settings, initial user and so on
  • This ISO will be generalized meaning it is hardware independent and can be used to install Windows on any computer capable of running Windows 10, regardless if the machine is a legacy BIOS machine with MBR partitioning, or a UEFI machine with GPT partitioning
  • The ISO image will be bootable on both BIOS / MBR and UEFI / GPT systems

NOTE: This post will show how to use a virtual machine to create the ISO. All virtual machine references and instructions in this tutorial apply to Hyper-V, available in Windows 10 PRO, Education and Enterprise editions. Oracle VirtualBox and VMware users might need to consult their preferred virtualization platform’s documentation if instructions can’t be used as is.
Everything in this instruction can be made in each edition of Windows 10 (in Home and Single Language editions using a third party virtualization platform) with native Windows tools and programs, apart from Windows Deployment and Imaging Tools, part of Windows 10 Assessment and Deployment Kit (ADK) needed later in the post. The ADK is a free native Microsoft tool, downloadable directly from Microsoft.
If you will do this on a Hyper-V virtual machine (which is the recommended method), make sure to set the new virtual machine to use Standard Checkpoints instead of default Production Checkpoints. You can do this in virtual machine’s settings:

Name:  image.png
Views: 109048
Size:  81.9 KB
Use Standard Checkpoints
Virtual machine generation is irrelevant, you can use Generation 1 or 2 as you wish

This method will produce an ISO image which can be compared to any original Windows 10 ISO you download from Microsoft, apart from the fact that it already contains pre-installed software according to your choice. It will also contain a customized and personalized default user profile, the base Windows uses whenever a new user profile will be created.

A customized default user profile means that whenever a new user account is created, all customizations (Start tiles, File Explorer & desktop icon and view settings, colors, wallpaper, theme, screensaver and so on will be applied to new user profile instead of Windows defaults.

Installation using this ISO will take somewhat longer than using a standard ISO because it not only contains full Windows setup, but also the pre-installed software. Notice that depending on how much space pre-installed software takes, you might not be able to burn this ISO to a standard 4.7 GB DVD disk but have to use a dual layer disk or a USB flash drive instead. My customized image came out to be about 8.5 GB in size.

The ISO created will include no user profile folders, personal user data and files.

This ISO image can be used on any hardware setup capable of running Windows and can be shared, subject to people you share the ISO with have valid licenses and / or activation keys for both Windows 10 and pre-installed software.

System Preparation Procedure

  • Download the Windows ISO Installation tool from Microsoft
    • Use this TOOL to download the ISO and create the installation media
  • Install Windows 10 on your VM using the downloaded ISO

NOTE: The normal Windows Download from the link above will download Windows 10 Professional. You will need a key for the installation to upgrade to Enterprise Edition and you will need to be able to activate the copy of Windows to be able to save the customizations you create for your ISO.

  • Boot into Windows 10 and do the following:
    • Activate the Windows Edition your are installing with your key. You will require internet connectivity. I needed Enterprise Edition so I changed the Product Key In Settings to upgrade it from Professional.
    • Install your preferred software, customize and personalize Windows, remove / add Start tiles as you wish, and set your preferred group policies (group policies not available in Home and Single Language editions). Do not run any program you install!
    • Update all software and run Windows Update to get all the latest updates for the image.
    • Notice that software installed now will be included in ISO install media, and will be pre-installed for all users on each computer you install Windows to using this custom ISO.

NOTE: If Windows on your reference machine is not activated, you cannot personalize it. In this case you need to modify Windows theme (wallpaper, screensaver, colors, sounds) as you wish on another, activated Windows 10 machine, save the theme as a theme file, copy it to inactivated reference machine and apply (double click).
Also notice that Edge as well as other UWP apps do not work when signed in to built-in admin account. If you need a browser to download software you have to use a third party browser or Internet Explorer. IE can be started from Run dialog by typing iexplore and clicking OK.

  • Open an elevated command prompt and enter the following:

Windows will now restart in Audit Mode using built-in administrator account. You will see a Sysprep prompt in the middle of display:

Name:  image.png
Views: 107212
Size:  73.1 KB
Sysprep Program Window
Leave it open for now
  • Open Notepad, paste the following code to it, make the necessary changes to customize the installation, and save it as
    File name: unattend.xml (exactly this name!)
    Save as type: All files (important!)
    Save in folder: C:\Windows\System32\Sysprep

  • When Sysprepping with the Generalize switch, which we will soon do, the component CopyProfile being set to be TRUE in answer file has a small issue or rather a small inconvenience: it leaves the last used user folders and recent files of built-in admin to end user’s Quick Access in File Explorer.
  • To fix this, we need to reset Quick Access to default whenever a new user signs in first time. It requires the need to run a small batch file at first logon of new user, and then remove the batch file itself from user’s %appdata% so Quick Access will not be reset on any subsequent logon.
  • To do this, open an elevated (Run as administrator) Notepad (Notepad must be elevated to save in system folders), paste the following code to it, save it as:
    File name: RunOnce.bat (or any name you prefer, with extension .bat)
    Save as type: All files (important!)
    Save File in folder: %appdata%\Microsoft\Windows\Start Menu\Programs\Startup

  • Delete all existing user accounts and their user profile data (Option One in this tutorial)
  • You are currently signed in using Windows built-in administrator account. In File Explorer, open C:\Users\Administrator folder and check that all user folders are empty deleting all possibly found content
  • Run Disk Clean-up, selecting and removing everything possible (tutorial)
  • When the disk has been cleaned, create a checkpoint of the VM from Hyper-V Manager. Right Click VM > Click Checkpoint
Manual Checkpoint from Hyper-V Manager
  • In Sysprep dialog still open on your desktop, select System Cleanup Action: Enter System Out-of-Box Experience (OOBE), select Shutdown Options: Shutdown, select (tick the box) Generalize, click OK:
Name:  image.png
Views: 110675
Size:  14.0 KB
Sysprep Selected Options Before Shutdown

Sysprep will now prepare Windows, shutting down machine when done. LEAVE THE VM OFF AND DO NOT RESTART IT! Now, we continue to the Image Creation section.

Name:  image.png
Views: 110209
Size:  3.4 KB
Sysprep Preparing the Machine

Image Creation Procedure

  • On your Hyper-V Host machine, open Disk Management
  • Select Attach VHD from Action menu:
Name:  image.png
Views: 109944
Size:  18.3 KB
  • Browse to and select your reference virtual machine’s VHD / VHDX file. If you have any checkpoints (AVHD / AVHDX files) created on this vm, select the one with most recent time stamp. Note that you have to select show all files to be able to see checkpoint AVHD / AVHDX files:
Click image for larger version.   Name:	image.png 
Views:	987 
Size:	94.6 KB 
ID:	113169
Select the most recent time stamped file
  • Select the check box labeled Read-only (this is very important!), then click OK:
Name:  image.png
Views: 110148
Size:  15.3 KB
BE SURE TO SELECT READ-ONLY

IMPORTANT: Forgetting to select Read-only will especially when mounting a checkpoint AVHD / AVHDX file make it unusable for Hyper-V! You will NOT be able to boot your VM and could corrupt it should you have write access on the mounted VHDX file.
Windows mounts the virtual hard disk, and all of its partitions, as separate disk. In case of an MBR disk it even mounts the system reserved partition.

Name:  image.png
Views: 110574
Size:  6.3 KB

NOTE: In the above picture the Windows system partition for the reference VM is drive K:

  • Open the Windows system partition VHD to be sure that’s the one where Windows is installed, note the drive letter your host assigned to it.
  • Open an elevated Command Prompt, enter the following command to create a new install.wim file:

NOTE: D:\install.wim path in this case is the drive and directory where you want to save the image file. K:\ path is the capture path with subfolders of the drive you want to image FROM

Click image for larger version.   Name:	image.png 
Views:	907 
Size:	26.5 KB 
ID:	113172
dism command

NOTE: The name given in /name switch in above command is irrelevant as we will name the ISO later on, but is needed for the command to run. Use any name you want to for the switch parameter.
The image process will take time, go get something to eat as I did. On my high end Hyper-V server this takes over 20 minutes, the first half of it without showing any progress indicator whatsoever. DISM works somewhat faster if you don’t use optional switches /checkintegrity and /verify but it is not recommended that you to create install.wim without checking its integrity and verifying it.

  • When completed capturing the image, detach the VHD / VHDX or AVHD / AVHDX file from host by right clicking it in Disk Management and selecting Detach VHD:
Name:  image.png
Views: 109850
Size:  28.5 KB
Detach the VHDX

ISO Image Creation Procedure

  • Mount the original Windows 10 ISO you downloaded in the first step to a Virtuial Drive on your Hyper-V Server Host.
  • Copy its contents (everything) to a folder you create on one of your Hyper-V host’s hard disks:
Click image for larger version.   Name:	image.png 
Views:	1221 
Size:	463.8 KB 
ID:	125114

I named the folder ISO_Files and placed it on the D: drive where I had created the image from the previous section. Alternatively, you can copy the contents of a created Windows 10 install USB or DVD to the ISO_Files​ folder.

  • Browse to your custom install.wim created earlier in previous section. Copy it to Sources folder under the ISO_Files folder, replacing the original install.wim in that directory:
Name:  image.png
Views: 110062
Size:  55.8 KB
Note

IMPORTANT: If the ISO you used when copying the files to the ISO_Files folder has been made with Windows Media Creation Tool, the ISO_Files\Sources folder contains an install.esd file instead of install.wim.
In this case you will naturally not get “File exists” prompt. Simply delete the install.esd file and paste your custom install.wim to replace it.
This will help reduce the overall size of the ISO and not confuse the installation process when ran.

  • Now, we download the latest Windows Assessment and Deployment Kit(ADK)for Windows 10: Windows ADK downloads – Windows Hardware Dev Center
    The full download for the ADK is about 7.5 GB but luckily we only need the Deployment Tools portion. So, unselect everything else except Deployment Tools and click Install:
Name:  image.png
Views: 106320
Size:  348.3 KB
Select Only Deployment Tools for the Installation
  • Once completed, you should have a folder within your start menu for the ADK Tools Installation under the folder Windows Kits. Start the Deployment and Imaging Tools interface program by Running the Program as an Administrator:
Name:  image.png
Views: 110029
Size:  38.8 KB
Right Click the Application and “Run As Administrator”
  • At the command prompt, type cd\ to bring your prompt to the root of the folder path you are on.
  • Type the following command to initiate creation of the ISO image file:

Replace three instances of d:\iso_files with the path to the ISO_Files folder where you copied Windows installation files. Notice that this is not a typo: first two of these instances are typed as argument for switch -b without a space in between the switch and argument. This is to tell the oscdimg command where to find boot files to be added to ISO.

Replace d:\14986PROx64.iso with the path where you want to store the ISO image. This is where you also name the ISO file what you want the file name to be.

Although the command seems a bit complicated, everything in it is needed. For more information about the oscdimg command line options, see: Oscdimg Command-Line Options

Click image for larger version.   Name:	image.png 
Views:	1467 
Size:	54.6 KB 
ID:	113179
Screenshot of the OSCDIMG command ran

You now have a completed ISO image ready for distribution to your machines. The overall process took me about 4 hours to complete with all the customizations that I did. Thanks again to Ten Forums for the article. I have provided references below for your convenience as well.

HAPPY IMAGING!!
PLEASE COMMENT!!

REFERNCES:
Create Windows 10 ISO image from Existing Installation
Open and Use Disk Cleanup in Windows 10
Download Windows 10
Windows 10 sysprep – how to skip entering product key
Windows ADK downloads – Windows Hardware Dev Center

The Windows Time Service, Hyper-V Hosts, and DCs that are VMs.

The sheer craziness of it all! I noticed that my clocks were off on my servers by FOUR minutes. I had originally set in group policy for the PDC emulator for my domain, a VM on one of my Hyper-V hosts, to get the time from the Public NTP hosts. I then configured a group policy to have all the other machines get their time from the PDC Emulator.

This was working great for me until I realized that my Hyper-V hosts were actually controlling the time of the VMs. They were also configured to get the time from the PDC Emulator, but essentially, due to how Hyper-V is configured, the PDC Emulator VM was getting the time from the Host. So, once the time got thrown off, everything went wacky on me!

I’d read through a couple of articles and found the configuration flaw of Hyper-V and the need for those servers to get their time from the external NTP hosts as well as be configured as NTP servers themselves. This totally went against my Group Policy configuration which caused the issue!

Luckily, I had a stand alone server that is a tertiary DC in the domain not running Hyper-V. I was able to get my time synced again properly after performing the following configuration.

  • I had to move the FSMO roles to the tertiary DC with the following cmdlet:

  • I then made sure the tertiary DC was syncing time correctly by running the following on that server:

  • I then removed the Group Policy Object for syncing the time source to the DC that I had linked to my Hyper-V Servers OU in Active Directory
  • Ran a gpupdate /force on the Hyper-V host to remove the policy there
  • I then had to reconfigure the Hyper-V hosts to be NTP Servers and clients that got their time from a public NTP server:

The one problem Hyper-V host that was syncing with the DC VM would not change settings via Group Policy nor through the w32tm cmdlet. I even went into the registry and tried to modify the following keys to make the changes stick:

The values would just not change, most likely due to the time not being synchronized. I had to reboot the server and then run through the process again in order for the changes to stick.

I did look at another article that said to do the following on the DC VM in order for time NOT to sync with the Hyper-V Host:

Go into Hyper-V console on the host machine, right-click on the client VM AD server, and select Settings. Once in here, on the left look under:

Management –> Integration Services
Untick Time Synchronization
Click Apply/OK

Virtual Machine Settings within Hyper-V

Things are running smoothly now. Please view the references at the bottom of the post. There are a couple of great articles about the Time Synchronization process with Hyper-V and why it needs to be setup the way I have it now. I wished I had read it before I originally set this up. I will post the article about getting group policy to handle the time sync process. Just remember, if your PDC Emulator is a VM, don’t sync it to a public NTP server. Sync it to your Hyper-V Host and have the Host sync publicly.
In the long run, I think it is a good design solution to have your Hyper-V hosts time synced to the Public NTP servers than having to remember to configure each VM DC you create to NOT time sync with the host. To each is own though, and one thing I learned from working Microsoft, there are multiple ways to get to the same goal that are technically sound methods.

THANKS FOR READING!
PLEASE COMMENT!

REFERENCES:
Setup of NTP on Hyper-V servers
Time Synchronization in Hyper-V
“It’s Simple!” – Time Configuration in Active Directory
NTP Circular Time Sync – Windows Server 2012 R2 / Hyper-V

Exchange Server Client Access URL Configuration Script

In my career, I have to be able to be efficient as most of my projects are on a time crunch schedule. Being able to quickly configure Exchange when setting up a server environment is crucial to the success of the project.

While still honing my skills in PowerShell, I was attempting to create my own script to help configure all of the Virtual Directories in one shot rather than go to each setting and configure them manually. It did not go very well, so as I do, I research and find great professionals that do great work in scripting so that I may learn from them.

In doing so, I found Paul Cunningham’s script that performs this. I took the following script and modified it to add the PowerShell Virtual Directory to it as I like to configure that as well.

***YOU CAN REM THE LINES OUT SHOULD YOU NOT WANT TO CONFIGURE THAT DIRECTORY***

Here is my version of the script:

NOTES:

  • PowerShell script to configure the Client Access server URLs for Microsoft Exchange Server 2013/2016. All Client Access server URLs will be set to the same namespace.
  • If you are using separate namespaces for each CAS service this script will not handle that.
  • The script sets Outlook Anywhere to use NTLM with SSL required by default.
  • If you have different auth requirements for Outlook Anywhere use the optional parameters to set those.
  • The script sets PowerShell to use Basic with SSL required by default.
  • If you have different authentication requirements for PowerShell use the optional parameters to set those.
  • PowerShell was added to the settings. Please be sure to REM those lines of code should you NOT want to configure the PowerShell Virtual Directory.

USAGE:

HAPPY SCRIPTING!
POSITIVE ENERGY!
PLEASE COMMENT!

REFERENCES:
Exchange Server Client Access URL Configuration Script
PowerShell Script to Configure Exchange Server Client Access URLs

Installing an ‘IP-less’ Exchange Server 2019 Database Availability Group

Yesterday, I posted on how Exchange now uses the Resilient File System (ReFS) to optimize and protect Exchange critical files. Another layer of protection is using a database availability group (DAG) for redundancy and is a necessary factor when designing an Exchange Enterprise Environment.
In this example, I will walk you through the installation of an Exchange Server 2019 DAG as I configured in my environment. This DAG will contain two Exchange Servers in the same site with a third Windows Server 2019 server being the File Share Witness (FSW).

Two Server Exchange DAG Configuration

For my configuration, I configured two identical Windows Server 2019 VMs (same procs, RAM, vhdx drives, partitions, etc…). I configured the Exchange Data Volume using ReFS and mounted them to the same folder on the C: Drive on each server. This is very important for replication to take place successfully when the databases are added to the DAG.


I next went to the Admin server where the FSW would be hosted and added the Exchange Trusted Subsystem Account to the local Administrators group on that server:

IMPORTANT!
Add the Exchange Trusted Subsystem Account to the Local Administrators Group on the FSW.

NOTE: The reason that this is an ‘IP-less’ DAG is that I’m creating a DAG with no cluster administrative access point (CAAP). The DAG has no IP address of its own, and no computer object in Active Directory. The main implication of this is that backup software that relies on the CAAP or backup operations won’t work. This option of an ‘IP-less’ DAG was first introduced in Exchange Server 2013 SP1/CU4, so by now any decent backup products should support this configuration. But you should always verify this with your backup vendor of choice. Also be aware that this is only supported for DAGs that are running on Windows Server 2012 R2 (or later).

Next, we create the DAG from Exchange PowerShell using the New-DatabaseAvailabilityGroup cmdlet. Now remember that since you are using the ReFS system for your database volumes, you will need to specify the -FileSystem parameter within the cmdlet to assure proper setup and replication of the data files.

Next, we add the Exchange Servers that hold the databases that will be replicated within the DAG:

The DAG will now show the two servers as Operational Member Servers:

The FSW Directory was created on the admin01 server when the DAG was created. We can verify that with the following cmdlet:

Next, we add the databases that we want replicated to the DAG as replicated databases. I want all my Databases on EX01 to replicate to EX02 and vice versa for the EX02 Databases. I want the activation preference to remain on the server that the databases were originally created on so I will use the -ActivationPreference parameter to accomplish that. I will go into more detail on Activation Preference in another post.

Now we verify that the Database Copies are healthy on each replication member using the Get-MailboxDatabaseCopyStatus cmdlet. You will see a Healthy Status on the replicated copies:

POSITIVE ENERGY!
KILL NARCISSISM!
HAPPY TROUBLESHOOTING!

REFERENCES:
Installing an Exchange Server 2016 Database Availability Group

Using the Resilient File System for Exchange Server

In my ongoing effort for becoming more knowledgeable on Exchange Server, I found that the preferred new file system for Exchange Databases and Log files is the ReFS.
ReFS is not that new. Microsoft’s Resilient File System (ReFS) was introduced with Windows Server 2012. ReFS is not a direct replacement for NTFS, and is missing some underlying NTFS features, but is designed to be (as the name suggests) a more resilient file system for extremely large amounts of data.

Support for ReFS with Exchange Server

From Exchange Server 2013 and upwards (which includes Exchange Server 2019 today) Microsoft supports the use of ReFS for Exchange servers, and in fact they now recommend it as the preferred file system for Exchange Server 2019, within the following guidelines.

For Exchange Server 2013:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features must be disabled for the database (.edb) files or the entire volume that hosts database files.
  • Hotfix KB2853418 must be installed.
  • For Windows 2012, the following hotfixes must be installed:

This means that you should continue to use NTFS for your operating system and Exchange Server 2013 installation volume, but you can consider using ReFS for the volumes hosting Exchange databases, log files, and index files.

For Exchange Server 2016:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features are recommended to be disabled.
  • For Windows 2012, the following hotfixes must be installed:

This means that you should continue to use NTFS for your operating system and Exchange Server 2016 installation volume, and it is recommended ReFS for the volumes hosting Exchange databases, log files, and index files.

For Exchange Server 2019:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features are recommended to be disabled.

This means that you should continue to use NTFS for your operating system and Exchange Server 2019 installation volume, and it is recommended ReFS for the volumes hosting Exchange databases, log files, and index files.

Creating an ReFS Formatted Volume

In Windows Server during the New Volume Wizard when you get to the step for configuring File System Settings change the file system from NTFS to ReFS.

exchange-server-refs

NOTE: Using the New Volume Wizard does not give you the option to disable data integrity at the volume level. To set it at the volume level itself use PowerShell when configuring new volumes. I found this out the hard way and am now re-configuring my volumes to disable the Integrity Streams.

I needed to create the mount point to mount the volume to:

I then got a list of my available disks:

In my case, disk 2 was the one I needed to format and change. I had to create a new partition and then format it:

Once formatted, I mount the volume to the Directory created earlier:

NOTE: Partition 1 on a disk is always reserved for system files on the drive volume. So the active partitions will always start at 2.

Lastly, verify that the partition is online and that the Integrity Streams are turned off:

Additional Considerations

When you are deploying an Exchange 2016 or 2019 DAG and using Autoreseed, the disk reclaimer needs to know which file system to use when formatting spare disks. So when, creating a DAG in Exchange PowerShell, make sure to set the -FileSystem parameter. For Exchange Server 2013 DAGs, manually format the spare volumes with ReFS.

More coming soon. I will post how I setup the “IP-less” DAG for my environment and got replication functional for my Exchange Databases.

REFERENCES:
Exchange 2013 storage configuration options
Exchange 2016 Preferred Architecture
Exchange Storage for Insiders: It’s ESE (Ignite video)
ReFS Exchange Server Volumes
Preparing ReFS Volumes for Exchange

Hyper-V General Access Denied error when trying to load a Virtual Hard Drive and start a VM

I was working on setting up a VM for my server farm and mis-configured one of the vhdx drives. I ended up having to delete that drive and recreate it in Hyper-V manager. When I did though, I received an error stating that I could not start the virtual machine:

An error occurred while attempting to start the selected virtual machine(s).
‘VMName’ failed to start. (Virtual machine ID ‘SomeID’)
‘VMName’ Microsoft Emulated IDE Controller (Instance ID ‘SomeID’): Failed to Power on with Error ‘General access denied error’ (0x80070005). (Virtual machine ID ‘SomeID’)
‘VMName’: IDE/ATAPI Account does not have sufficient privilege to open attachment ‘C:\Users\Public\Documents\Hyper-V\Virtual hard disks\DiskName.vhdx’. Error: ‘General access denied error’ (0x80070005). (Virtual machine ID ‘SomeID’)
‘VMName’:  Account does not have sufficient privilege to open attachment ‘V:\Hyper-V\Virtual hard disks\DiskName.vhdx’. Error: ‘General access denied error’ (0x80070005). (Virtual machine ID ‘SomeID’)

Causes

Each virtual machine is started using a virtual machine account. The virtual machine account needs read and write access to the .vhd/.vhdx file, but if the file has just been copied from somewhere then it most likely lacks the necessary file permissions.
That happened in my case because I had just created the vhdx drive and did not create it from the VM itself. I just attached it to the VM. So, when I booted the VM, it gave the error.

Remediation

There are a few ways that you could remediate the issue. The simplest way, if it is a new VM, is to remove the drive in the VM settings and then re-create it from scratch. That is what fixed it for me.
Another way is to add the VM GUID to the permissions so that it can access the vhdx file properly:

  • If you don’t already have the Hyper-V Manager error dialog open (“An error occurred while attempting to start the selected virtual machine(s) …”) then try to start the virtual machine now. You need the error open.
  • Click “See details”. This will show additional details, and will look something like:

‘PC-Name’ failed to start. (Virtual machine ID B9C4F7D4-0009-4BE2-90FB-9D60B1A06BDD) ‘PC-Name’ Microsoft Emulated IDE Controller (Instance ID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX): Failed to Power on with Error ‘General access denied error’ (0x80070005). (Virtual machine ID B9C4F7D4-0009-4BE2-90FB-9D60B1A06BDD)
‘PC-Name’: IDE/ATAPI Account does not have sufficient privilege to open attachment ‘E:\Hyper-V\PC-Name\Virtual Hard Disks\MyVHD.vhdx’.
Error: ‘General access denied error’ (0x80070005). (Virtual machine ID B9C4F7D4-0009-4BE2-90FB-9D60B1A06BDD)
‘PC-Name’: Hyper-V Virtual Machine Management service Account does not have sufficient privolege to open attachment ‘E:\Hyper-V\PC-Name\Virtual Hard Disks\MyVHD.vhdx’.
Error: ‘General access denied error’ (0x80070005). (Virtual machine ID B9C4F7D4-0009-4BE2-90FB-9D60B1A06BDD)
Where PC-Name will be the name of your virtual PC. The long sequence of letters and numbers (in my case above “B9C4F7D4-0009-4BE2-90FB-9D60B1A06BDD”) is the Virtual Machine ID. This number is significant and you need it to fix the problem.

  • On the host server open an elevated command prompt.
  • Enter the following:

You will need to substitute the path to the vhd/vhdx file – you can obtain this from the original error message, and the Virtual-Machine-ID that you obtained from the “See details” part of the error.

So the line for me was:

NOTE: If you get the message “Failed processing 1 files” then check the virtual machine ID.

  • Now try to start the virtual machine. The error should no longer be present.

There is also a PowerShell Gallery script that is supposed to remediate this issue:

http://www.ntsystems.it/page/PS-Restore-VMPermissionps1.aspx

I haven’t tried it but it looks as it would work. Please review and leave a comment should you have issues with the script.

HAPPY TROUBLESHOOTING!
PLEASE COMMENT!
POSITIVE ENERGY!

REFERENCES:
Resolved: Hyper-V General access denied error when trying to load a Virtual Hard Drive
Restore-VMPermission
Virtual machine fails to start with General access denied error / Account does not have sufficient privilege to open attachment

Importing User Photos to Office 365 in bulk for your company.

In a previous post, I showed how you could update one user’s photo for their Outlook and AD profiles via PowerShell. In this post, we will explore how to do this for your entire organization via PowerShell to Office365.

NOTE: I have not tested the scripts as I do not have enough mailboxes in my O365 tenant along with not using a ‘.’ in my alias. If the scripts are incorrect, please inform me with the correction and I will update accordingly.

Please make sure that your photos are reviewed before posting, and try to keep the file size of the photos to a minimum. In Office 365, there exists a limitation for the user photo not to be more than 10 KB in size, but I will show you how to get around that limitation.

Having a user photo for each of your users is very beneficial as it personalizes each account to a face in the company. The user photos can be viewed in below locations:

  • Outlook Web Access
  • Contact Card
  • Thumbnail in emails
  • Outlook Client
  • Yammer
  • Lync Client
  • SharePoint (People Search / Newsfeed)

Steps to take:

  1. Remove the 10KB photo size limitation in Exchange Online
  2. Prepare a folder with all users photos
  3. Update the profile photos via a PowerShell cmdlet.

Connect to Exchange Online with the RPS Proxy Method to remove the 10K size limitation

NOTE: In the PowerShell cmdlet above, we connected using a different proxy method. This was to overwrite the limitation of uploading the images with size more than 10KB. Using the different proxy method (/?proxyMethod=RPS ) to connect to Office 365 in the above cmdlet accomplishes this.

Prepare a folder locally and place all the photos in that folder

Create a folder named C:/UserPics and make the filename of each photo be the username of that particular user. (i.e. llingerfelt.png)
The below script should be able account for aliases that have a ‘.’ in the id as well. (i.e. lance.lingerfelt)

NOTE: From my research, there is no set photo type that is required for the photo. My suggestion would be to keep the photos .png for size constraints while maintaining picture clarity.

Update the profile pictures via PowerShell

Create the following script and name it Photos-Update.ps1

Run Photos-Update.ps1 and the script should upload the photos to Office 365 and apply each photo to the corresponding user.

NOTE: If you’re still having some issues with the alias having a ‘.’ in the name, you can also configure the Photos-Update.ps1 script in this manner to get that working properly:

HAPPY SCRIPTING!
PLEASE COMMENT!

store.ldlnet.net
LDLNET LLC! Your Source for Pofessional IT Services!
www.servermonkey.com
Contact ServerMonkey.com for your IT Hardware Needs!

REFERENCES:
How to import Office365 User photos over 10KB & without CSV in bulk

Set the profile pic for a single Exchange user via PowerShell

I wanted to update my picture within my Outlook profile and AD account really quickly without having to go through OWA to do so. I found this cmdlet that will allow for that picture to be changed very quickly via Exchange PowerShell.

NOTE: This can be done with On-Premises Exchange and Exchange Online PowerShell

Old picture within my account

First, download the picture you want to use to the computer that you want to run the cmdlet from. Also, make sure the picture is cropped and centered prior to running the cmdlet. I saved the pic to C:\temp for my scenario. The best format to use would be jpg. I named the file User1_Profile.jpg

Next, open Exchange PowerShell on the computer you saved the pic to and run the following cmdlet to change the photo:

Once completed, the Outlook client should be closed and reopen so that the new picture is visible in the profile.

Picture change completed

I will post how to perform this for multiple users for Exchange and Office365 in a later post.

REFERENCES:
Set User Photo with Exchange PowerShell

Purging Soft Deleted mailboxes from Exchange Server

If you’re a seasoned administrator, you have knowledge that in Exchange, the database settings will allow you to set the deleted mailbox retention. The default is 30 days, but sometimes you need to purge all those deleted mailboxes to do some ‘spring cleaning’ as it were. Note that doing these cmdlets does not change the ‘Whitespace’ of the database or the size. In my case, I had to purge everything of a toxic individual that was tainting my network much to my disappointment and did the following to complete that task.

The following cmdlet will seek all Soft Deleted mailboxes within the database you select and manually purge them from Exchange.

Now, should you only want to remove one mailbox, you will need to get the GUID of that Soft Deleted mailbox first so that you can enter it for the identity parameter.

You can also preform a similar task for a disabled mailbox:

You can perform the task on all disabled mailboxes for that database as well:

NOTE: I would be very careful when performing either of these cmdlets as they will completely purge the mailboxes from the schema. If these cmdlets assist you with your ‘spring cleaning’, I will have been happy to assist.

HAPPY PURGING!
PLEASE COMMENT!
IGNORANCE IS NOT BLISS!

References:
Purging Deleted Mailboxes on Exchange 2013

Server Monkey
LDLNET LLC’s Preferred Server Equipment Hardware Vendor

Installation of Exchange Server 2019 on Windows Server 2019

I have realized recently that I am an Exchange Messaging Professional, but yet, I have not posted the methodology of how I install an Exchange Server Mailbox Role. So here it is!

Install Windows Server 2019

Exchange Server 2019 requires Windows Server 2019 to run. For my environment, I haven’t necessarily need to follow all the enterprise level design aspects of database numbers to mailbox size ratios, number of servers, front/back end configurations, DAG Implementation, etc… If you want or need to delve into that realm, you can go here. I have need for a single server with only a few databases for a small number of mailboxes, so I am approaching it from that standpoint.

So first, in Hyper-V, I configured my VM with the following specifications:

Processors: 2 procs with 2 cores each – 4 Virtual Processors Total
RAM: 32GB with dynamic memory enabled optional
Drives: 2 .vhdx drives of 120GB each (OS / Exchange Data)
CD: Windows 2019 ISO
Default Settings for the rest of the VM Settings

Next I installed Windows Server 2019 Datacenter with the GUI! You can install it on Server Core if you wish. That information can be found in this link.

I ran through the setup of Windows and installed the OS on my first vhdx drive. I booted up, set the local admin password, and logged in. Once in Windows, I went to the Local Server Settings in Server Manager and configured the following settings:

Set the Date, Time, and Time Zone. (Once in the Domain, this would sync through Group Policy)
Set IE ESC to allow Administrators to have full IE access.
Set Remote Desktop Settings to gain RDP access. (This would be locked down with Group Policy as well once on the Domain)
Set the IP Settings to Static Settings. (DNS Servers, Gateway, WINS, etc…)
Join the server to the Active Directory Domain.
Reboot the VM Server.
Logon to your Domain.
Configure Windows Update Settings. (I have WSUS through Group Policy, this was configured automatically upon reboot)
Download and install all Windows Updates for the server. Then Reboot.
Open Disk Management and configure the secondary vhdx drive to be your Exchange Data Drive.
I configured the drive to be a mounted folder ‘C:\Exchange\Data’ rather than another drive letter as that seems to be the more accepted form of installation for the data drive these days. That is based on the multiple configurations that I have seen for Exchange through experience in Enterprise environments. Again, to each is own and depending on you design specifications, you might want to do that differently.

Next, we need to install the prerequisites for the Exchange Mailbox Server. I have always used practical365.com to get the PowerShell script to install the prerequisites, but couldn’t find the article this time. Great site though! Instead, I got the information and ran the following from an elevated PowerShell Session locally on the server:

As part of the prerequisites you will need to install the following packages onto the server as well:

UCMA Runtime Install
Visual C++ Redistributable Packages for Visual Studio 2013

Once completed, you can begin the install of Exchange. If this is your first Exchange 2019 Server in your Organization, then you will need to run the following to update the Forest, Schema, and Domain so that Exchange will install properly:

NOTE: If you run into Prerequisite issues with the installation due to a “pending reboot”, check out my blog post for information on remediation of that issue.

Now that the environment is prepared for Exchange, you can actually begin the installation. I wanted to make my default database and logs folder to be on the Exchange Data volume that I created, so I included those settings in the setup command. Please look at the reference to the setup.exe switches for more information on that. Here is the command:

Setup should go through the installation via the PowerShell window and complete successfully. Reboot the Exchange Server, then you can then logon to the Exchange Admin Center and begin the process of configuration of how you need to integrate the Mailbox Server into your Server Farm. That configuration is for a later post.

PLEASE CHECK BACK FOR UPDATES!
PLEASE COMMENT!

References:
UCMA Runtime Install
Visual C++ Redistributable Packages for Visual Studio 2013
Install Exchange Server 2019 on Windows Server 2019 Core
Exchange Server Design Planning
Use unattended mode in Exchange Setup
Practical365 on Exchange 2019

Removing Hidden Devices in Device Manager

As you may have knowledge of, if you are reading my blog. I am currently migrating off of VMWare to Hyper-V. Now, as I convert my machines to Hyper-V, it uses a totally different driver for the Network Card. I am having to rebuild the NIC settings within windows to setup the NIC for the Hyper-V VM to get the machines on the network properly again. The VMWare NIC disables and hides the NIC from the VMWare driver in Device Manager.

What this does is make Windows think it has two active network cards, even though one is disabled and removed/hidden in device manager. So, to clean things within Windows, I have to perform the following procedure to remove the hidden device:

Open PowerShell as Administrator
Next, type the following cmdlet and press Enter:

Next, open Device Manager from the PowerShell Session:

When the Device Manager GUI opens, click the View menu
Click 
Show Hidden Devices
Go to the Device that is hidden, in my case the Network Adapter
Right-Click the Device and select Uninstall

Close the Device Manager GUI and PowerShell session

This cleaned the old hardware drivers off the system and allowed the current Hyper-V NIC to be the only one installed.

HAPPY TROUBLESHOOTING!
PLEASE COMMENT!

How to transfer FSMO Roles using PowerShell

A rare weekend post for me! HA! I am currently migrating my server environment from VMWare 6.7 to Server 2019 Hyper-V. I have a separate standalone box that I use for my VM backups and as a tertiary DC. Since I had to shut down my VMs in order to convert them, I needed to quickly move my FSMO roles from the DC Virtual Machine to the Standalone box so things would stay running.

I found this great article on how to do that quickly through PowerShell since it is a pain to go into ADUC, ADDT, and setup an MMC for the Schema snap-in.

When you create a domain, all FSMO roles assigned to the first domain controller in the forest by default. You can transfer FSMO roles from one DC to another both the Active Directory graphics snap-ins and the PowerShell command line. Moving FSMO roles using AD PowerShell has the following benefits:

  • You do not need to connect with a MMC snap-ins to the future role owner;
  • Transferring or seizing FSMO roles does not require a connection to the current or future role owner. You can run AD-PowerShell module cmdlets on a Windows Client or Server running RSAT Tools;
  • To seize the FSMO role (if the current owner is not available), it suffices to use an additional parameter -force.

Import the Active Directory Module Into PowerShell:

To get the current forest level FSMO role owners (Domain Naming Master and Schema Master roles) you can use the following PowerShell cmdlet:

To view domain-wide FSMO roles (Infrastructure Master, PDC Emulator and Relative Identifier Master roles):

Transfer FSMO Roles using PowerShell

To transfer FSMO roles between Active Directory domain controllers, we use the PowerShell cmdlet:
Move-ADDirectoryServerOperationMasterRole

To use the Move-ADDirectoryServerOperationMasterRole cmdlet, you must meet the following requirements:

  • There must be at least one DC with a version of Windows Server 2008 R2 or higher
  • PowerShell version 3.0 or newer
  • Active Directory module (2.0  or newer)

NOTE: Unlike the Ntdsutil.exe utility, the Move-ADDirectoryServerOperationMasteRole cmdlet can be performed from any domain computer to migrate the Operations Master roles if you have the appropriate rights (Domain admins and Enterprise Admins).

Import the AD Module:

I needed to move all the roles from one server to the other, so, I ran the following to do so:

NOTE: To simplify the command, you can replace the names of roles with numbers from 0 to 4. The correspondence of names and numbers is given in the table:

PDCEmulator0
RIDMaster1
InfrastructureMaster2
SchemaMaster3
DomainNamingMaster4

So, by having knowledge of these numbers, you can simplify your cmdlet:

NOTE: In the event that the current owner of one or all of the FSMO roles fails, the forced transfer of FSMO roles is performed by the same command, but with the -Force option. Also, after the FSMO roles have been seized, the domain controller from which the roles was seized should never be connected to the domain. You will need to preform a metadata cleanup of the Schema before even thinking about putting that failed server back into production.

Once completed, I ran the previous cmdlets of Get-ADForest and Get-ADDomain to verify that the FSMO roles moved to the destination server.

As of now, my conversion to Hyper-V is going smoothly, although it takes quite a bit of time to convert the hard disks. Thanks again!

HAPPY TROUBLESHOOTING! KEEP SCRIPTING!
PLEASE COMMENT!

Reference:
How To Transfer FSMO Roles Using PowerShell

Generate Automated E-Mail From Performance Counter Alert

Recently, I was trying to load this IT Blog and noticed that the page took a very long time to load properly. So, first thing I did was logon to my Web Server to check things and saw that the Performance was very slow, almost to a halt. When I was able to pull up Server Manager, I saw the following:

The Performance Alert was pegged at 99+ alerts. I went into the alerts for the performance and saw the following:

My CPU Usage was pegged at 100% for many hours. As I was having issues with the server and could not dig any deeper to find what was running to cause the error other the the w3wp process, I decided to reboot the server.
Rebooting did resolve the issue at hand, but I needed a better way to alert myself when the processor pegged at 95% or more over at least a steady two minute period. That way I could keep the websites going and restart the server if necessary due to an issue with a process pegging the CPU.

I don’t have SCCM or any robust alerting software like we have a my employer, but not to worry, Windows Server has a built in method to generate a task using Task Scheduler when the Performance Counter is above threshold for the time period that you set.

Step 1 – Create your PowerShell Script

The first thing that you need to do is write a Powershell script that when run can send an email. While researching this I discovered many ways to accomplish this task, feel free though to experiment and use what is right for your environment.

Here is an example of a Powershell script that when used in conjunction with Task Scheduler and Perfmon can send an email alert automatically when any user defined performance counter threshold condition is met. In my environment I set this to C:\PowerShell\AlertEMail.ps1

If you notice, this Powershell script takes four arguments and assigns them to variables used in the output. It also saves the computer name to a variable which is also used as part of the output. By doing this the script can be used to send an email on any Perfmon Alerting Counter and on any server without additional customization of the script.

Step 2 – Setup a Scheduled Task To Be Run

In Task Scheduler we are going to Create a new Task as show in the following screen shots.

Create Task

Give the Task a name, you will need to remember it for the next step. Make sure the task is run with an administrative account and is run whether the user is logged on or not.

There are no Triggers to setup here. This Task will actually be triggered through the Perfmon Counter Alert we will set up in Step 3.

On the Action tab you want to define a new action. The action will be to Start a Program and use the following inputs, please adjust for your specific environment.

Program/Script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Add Arguments: -File C:\PowerShell\AlertEMail.ps1 $(Arg0)

Next, set your Conditions and Settings Tabs to the following settings as I did for this purpose. If need be, you might need to set them differently, but these settings should work just fine.

Step 3 – Create the Alert Trigger in Performance Monitor

Open Performance Monitor and Create a new Data Collector Set under Data Collector Sets > User Defined

Name the Alert (This one is for the Processor Usage)
Assign it as a Performance Counter Alert

I needed to monitor the Total % Processor Time for all processors on the server combined, so I set the counter to monitor from the available counters.

I then set the threshold to Alert when the processor is over 95% with Processor Time. I also let it run as the System account, and then saved the settings:

Once you have created the Data Collector Set go into the Properties of it and make sure the Alerting threshold and Sample Interval is set properly for each Performance Counter. Keep in mind, if you sample every 10 seconds then you should expect to receive an email every 10 seconds as long as the performance counter exceeds the threshold you set. I set my Sample Interval to 2 minutes so that if the percentage stays above 95% over that long of a time interval, I should take a look at the server and see what process is pegging the CPU:

If you select the Log an entry in the application event log don’t expect to see any entries in the normal Application event log. It will be written to the Microsoft-Windows-Diagnosis-PLA/Operational log in the Application and Services log directory.

Finally, we have to set an Alert Task that will trigger the Scheduled Task (AlertEMail) that we created in Step 2. You see that we also pass some of the Task arguments which are used by the PowerShell script to customize the email with the exact error condition associated with the Alert:

Once the Data Collector is configured properly you will want to start it.

If you configured everything correctly you should start receiving emails any time that alert threshold is surpassed. I also manually ran the script once from PowerShell with no Argument Data just to make sure the script would generate the email to my E-Mail Address.

Step 4 – Set the Data Collector Set to run upon server startup.

You now have one more step. Whenever you reboot a server the Perfmon Counter Alert will not start automatically. In order to survive a reboot you must run the following at a command prompt.

Note: “Processor Usage Alert” referenced in the script below is the name of my user defined Data Collector Set.

I now have a simple way to monitor my Web Server should this event happen again with the processor being pegged over a two minute period.

PLEASE COMMENT! I LOVE IMPROVEMENTS TO MY SCRIPTS! HAPPY TROUBLESHOOTING!

Reference:
Setup E-Mail Generation from a Performance Counter Alert

Check Windows Updates Installed via PowerShell

I had an issue last night where a server lost Secure Channel Connection to the PDC Emulator (NETLOGON Event IDs 5719 and 5783). All tests to test the secure channel via PowerShell were failing. (i.e. nltest or Test-ComputerSecureChannel cmdlets) The server essentially needed to be rebooted. I had a dumb dumb in my brain and forgot to check to see if there were any pending Windows Updates, because those need to be installed at the proper time and to a schedule. So, when I ran the following command to reboot:

The Windows Updates were installed inadvertently which could have caused even more issues if they were NOT approved or caused another failure on the server. TO NOT DO THIS IN THE FUTURE, remember to run the following command to shut off the Windows Update Service BEFORE initiating the reboot of the server:

But, the deed was done. NOW, I had to find out quickly what updates WERE installed via PowerShell so that I could alert the proper folks and give them a heads up on possible issues. Luckily, the server did NOT have any issues and the initial problem with NETLOGON was resolved. Here is the command I ran to find out the installed hotfixes filtered by today’s date:

Here was the Output:

Caption=http://support.microsoft.com/?kbid=4480960 
CSName=DC01
Description=Security Update 
FixComments= 
HotFixID=KB4480960 
InstallDate= 
InstalledBy=NT AUTHORITY\SYSTEM 
InstalledOn=2/26/2019 
Name= 
ServicePackInEffect= 
Status= 

Caption=http://support.microsoft.com/?kbid=4480965 
CSName=DC01 
Description=Security Update 
FixComments= 
HotFixID=KB4480965 
InstallDate= 
InstalledBy=NT AUTHORITY\SYSTEM 
InstalledOn=2/26/2019 
Name= 
ServicePackInEffect= 
Status=
 

Since there were no issues, I was able to resolve the incident. I did notify the account team though of the inadvertent installation so that they could revert the changes if necessary.

Remember, troubleshooting to resolution is a methodical process, and when in an enterprise environment, you MUST be aware of all factors of change process, even when the resolution is a simple reboot of the affected server.

HAPPY TROUBLESHOOTING!
I LOVE COMMENTS! THANKS FOR READING!

References:
Methods of generating installed updates via PowerShell
Check Windows Update History via PowerShell
Disable or Bypass Windows Update Installation During Reboot/Shutdown of a Server

Website Availability Checker Script

We get alerts from time to time that SharePoint and Skype Websites might not be loading correctly or not at all. The following is a script that will generate a report, with time and date, showing the availability of the websites in question and the time it takes to load them.

First off, you want to create a text file called URLList.txt that will have the list of websites you want to check. I save the file in the C:\PowerShell directory but you can change the path to where you need it. If this happens often, having this list prepared will help you evaluate the environment quickly and allow you to troubleshoot if necessary efficiently.

URL List Text File

Here is the script:

Here is your output:

PowerShell Output
The report shown in your web browser

HAPPY TROUBLESHOOTING!
MAKE THIS SITE BETTER!
PLEASE COMMENT!

Moving mailboxes to O365 via PowerShell in Hybrid Configuration

As many of you have knowledge, I am studying for my MS-202 Exam. And, part of the knowledge needed is to be able to migrate mailboxes between on premises and Exchange Online through PowerShell. Here are the steps for the scenario to move a mailbox from on premises to O365:

1. Connect to Exchange Online via PowerShell

If you have read my previous post: Connect to All PowerShell Modules in O365 with one script
You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.

2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.

3. Move a single mailbox.

In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved.
Now we initiate the move with the cmdlet. Similar to what you would do in the GUI, this simple mailbox move cmdlet initiates the move request. It has most of the same parameters as a local move request including BadItemLimit, LargeItemLimit, AcceptLargeDataLoss, etc…
 
Use the following LINK for documentation on the New-MoveRequest cmdlet.

Now with all migration projects, we expect to have to move multiple mailboxes in a single batch. The following will show the process for moving mailboxes in bulk from on premises to O365:

1. Connect to Exchange Online via PowerShell

If you have read my previous post: Connect to All PowerShell Modules in O365 with one script
You should have all the settings needed to connect your PowerShell to O365. Note in this scenario, that all these cmdlets will be run from O365 PowerShell and will be monitored from O365 by either PowerShell or the Exchange Admin Center. You will not be able to monitor the moves from On-Premises.

2. Provide your on premises Migration Administrator credentials as a variable for your cmdlet.

3. Move multiple mailboxes in a single batch.

In your hybrid configuration you should be doing directory sync with O365/Azure and the accounts should be available in the cloud showing that they are synced with AD. This also assumes that you have your MRS Proxy endpoint enabled, which can be done by the HCW. Also, make sure you have your licensing available for your mailboxes. From my knowledge, you can assign your license to the account in the cloud before moving, especially if you have a particular license that you need to assign the account. Other than that, moving the mailbox will assign an existing license that is available that includes an Exchange Online mailbox feature when the mailbox is moved.

This time you want to create a CSV file using the alias or emailaddress as your header and then list the appropriate value for all the users in your batch group. Save the file locally as MigrationBatch01.csv or a name of your choice.

Use EMailAddress
 OR
 Alias as the header

Next you initiate the mailbox moves. When specifying the mailbox identity in the cmdlet, use the respective header in your variable declaration (either $user.EMailAddress OR $user.Alias)

Use the following LINK for documentation on the New-MoveRequest cmdlet.

GOOD LUCK WITH YOUR MIGRATIONS!
HAPPY TROUBLESHOOTING!

References:
Moving Individual Mailboxes to O365
Move Mailboxes in Bulk to O365
PowerShell Mailbox Migration to O365
Connect to all PowerShell Modules in O365 with one script
New-MoveRequest Microsoft Document

Exchange DAG Replication Problem: An established connection was aborted by the software in your host machine

I had an issue with a four node DAG where the DR site with two of the DAG members were having replication issues. It was only technically affecting one DAG Member though. The copy queue length was really high and the logs were not committing to the database. A Test-ReplicationHealth cmdlet test told that the copy queue length for the affected database copy was high. No other databases were affected as there were eight databases on this DAG Node. The issue was that the log files were not replicating properly to the one DAG member for that database, causing the log file drives on all the other DAG members to build and become full:

EX04 DAG member has high Copy Queue Length
The purple member (EX04) free space is different from the other three DAG members

Circular Logging was turned on, but since the db was NOT in sync, the logs could NOT truncate properly which rendered CL useless. What was being done to stave the issue was to suspend the database copy of the affected DAG member (EX04), then resume the copy. The logs would replay and commit to the database copy on the DAG member, but over a short period of time, the same issue would arise again, as shown in this graph:

You can see the other DAG members start dropping in free space

There were absolutely no errors in the Event Viewer showing this replication issue. After some research, I ran the following cmdlet showing a particular output parameter that gave me the actual problem:

Get-MailboxDatabaseCopyStatus DAG1DB01 | ft -a -wr Name, Status, IncomingLogCopyingNetwork

Output with the actual error listed for the DR DAG members.

The operative error here was: {An error occurred while communicating with server ‘EX01’. Error: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.} 

Now even though only EX04 was actually having problems with its log replication, both DR members EX03 & EX04 were having the same problem. Again, there were NO events in event viewer showing this issue. I next did some connectivity tests to EX01 from EX04 even though the error said there was an established connection that was broken.

Ping EX01 -f -l 1472

Now the -f states do NOT fragment the packet and send it as a whole to the destination.
The -l states the packet/buffer size you want sent. In this case 1472 bits.
By doing this, you are able to assure that a router or switch is NOT segmenting the packets, packet segmentation of replication logs can cause data corruption and replication issues.

That test passed successfully. I also did a trace route to assure there was no packet loss on the route to the replicating server. That test passed successfully.

I next checked the DAG Network to assure that all networks were working for replication. Now, in this scenario, there was only ONE DAG Network, there was NOT a separate Replication Network. I did not design the DAG and limitations most likely came into play during the design. From my experience, you setup a separate replication network for replication only, but if your network has enough bandwidth, and the design calls for simplification, you can use one DAG network in your design.

Get-DatabaseAvailabilityGroupNetwork | fl 

RunspaceId : a1600003-8074-4000-9150-c7800000207f 
Name : MapiDagNetwork 
Description : 
Subnets : {{192.168.1.0/24,Up}, {192.168.2.0/24,Up}} 
Interfaces : {{EX01,Up,192.168.1.25}, {EX02,Up,192.168.1.26},{EX03,Up,192.168.2.25}, {EX04,Up,192.168.2.26}} 
MapiAccessEnabled : True 
ReplicationEnabled : True 
IgnoreNetwork : False 
Identity : DAG1\MapiDagNetwork 
IsValid : True 
ObjectState : New 

All the DAG Network Members were up and not showing errors. I next did a telnet session to EX01 over the default DAG replication port 64327 to see if there would be any connectivity issues to EX01:

telnet EX01 64327

That test was successful and there were no connectivity issues to EX01 from EX04. Again, there was only ONE database out of eight that was having replication problems. After mulling over the problem, it was decided to restart the MSExchangeRepl service on EX03 AND EX04 since the error was present on both DAG members. We would then, suspend the database copy and resume the database copy on the affected servers.

Run on EX03:
Restart-Service MSExchangeRepl
Suspend-MailboxDatabaseCopy DAG1DB01/EX03 -Confirm:$False
Resume-MailboxDatabaseCopy DAG1DB01/EX03 -Confirm:$False

Run on EX04:
Restart-Service MSExchangeRepl
Suspend-MailboxDatabaseCopy DAG1DB01/EX04 -Confirm:$False
Resume-MailboxDatabaseCopy DAG1DB01/EX04 -Confirm:$False

After monitoring the databases and log drives, the issue was resolved and replication started functioning properly.

Log Drive Available Space Returned to Normal for DAG members

PLEASE COMMENT! I WELCOME SUGGESTIONS, TIPS, ALTERNATIVE TROUBLESHOOTING! HAVE A GREAT DAY!

Customize your Default PowerShell CLI Prompt

We all like to have our customization in our Windows Desktop. Custom colors, icons, wallpaper, etc.. Well IT guy/gal, why not have your PowerShell CLI the same way? I’ve looked around at a few blogs and got some ideas to share with you on customizing your PowerShell CLI.

Now, by default, Windows looks in the following directory for your customization file:

C:\Users\(username)\Documents\WindowsPowerShell

It looks for a file called profile.ps1 and will load that script every time you load PowerShell once it is customized.

You can construct the script within PowerShell ISE or your favorite editor. The following is how I programmed the script to customize my PowerShell prompt:

First, we want to clear the slate on the PowerShell CLI. 
I like to add my contact information for instance.

Second, I run a script that I came across and added to my customization.
It get’s me the weather for the local area that I am from. Click the link for details.

How To Uniquify Your PowerShell Console (Scroll to Getting the Weather)

Next, I customize the PowerShell Window Settings so that it is the size and shape that I want.
I set the Directory Location, The Colors, and The Window Sizes.

Next, I wanted to write some text in the window before I get my PowerShell Prompt.
I work at Avanade, so I wanted to put a welcome message, today’s date, and what PS version I was running just because I could.

Lastly, we actually configure the prompt. There is a lot of ways to do this and I will leave references for you at the bottom of this post so that you can get more details on the commands actually run.
I wanted to have a colorful prompt that stated the company motto and put the current time.
I also configured the Window Title at the top to show text, the current user, and the current directory with different colors as that is my expressive part of my brain. 🙂

Here is the script in its entirety:

Here is the final product when opening PowerShell:

Avanade Custom PowerShell Prompt
My Customized PS Prompt

HAPPY SCRIPTING!
HAPPY VALENTINES DAY!

References:
Customizing your PowerShell Profile
Get-Weather.ps1
Modify your PowerShell Prompt
PowerShell Basics: Console Configuration

Connect to all PowerShell Modules in O365 with one script

Let’s say you’re an admin that needs to connect to Office365 via PowerShell often. Now, there are many different websites or blogs that will show you how to connect to each session via PowerShell. That can cause a headache since you can end up having five different PowerShell sessions running in five different windows. You end up having to enter a username and password all those times, which can become time consuming.

I want to show you here how to combine all those sessions into one script where, if you’re security is tight enough on your computer, you don’t even have to enter credentials. This way, you can click on one icon and pull up all the O365 PowerShell commands that you’ll need to manage your organization.

First you need to download the following PowerShell Module Installation Files so that your PowerShell Database will have the correct modules installed:

Microsoft Online Service Sign-in Assistant for IT Professionals RTW
Windows Azure Active Directory Module for Windows PowerShell v2
SharePoint Online Management Shell
Skype for Business Online, Windows PowerShell Module

Next, we want to setup the CLI (Command Line Interface) to be too cool for school. I have learned it helps to have knowledge of how to customize the CLI window. You can do all of this in PowerShell ISE or Notepad, which ever you prefer. Here are the commands for the script that I use to setup the CLI:

Next, you want to set your Execution Policy and put in your credentials so that you won’t be prompted to enter the user credentials when you run the script.

NOTE: MAKE SURE YOU KEEP YOUR SCRIPT SAFE AS THE CREDENTIALS ARE VISIBLE WITHIN THE SCRIPT IN PLAIN TEXT!

You can, alternatively, set your script to prompt for credentials every time by using the following:

$LiveCred = Get-Credential

Here is that part of the script:

Now we get into the importing of the modules for each O365 service:

Get the MSOnline Module:

Connect to the MSOnline Service:

Connect to Azure AD PowerShell:

Connect to SharePoint Online PowerShell:
NOTE – MAKE SURE YOU CHANGE TO YOUR COMPANY NAME IN THE URL!!

Connect to Exchange Online PowerShell:

Connect to Skype For Business Online PowerShell:

Connect to the Security & Compliance PowerShell:
NOTE – This one I still get “Access Denied” when trying to connect. I have looked for an answer to that issue, but have not found one. Please comment with a link if you have an answer so that I can update this script!

Lastly, put in a note to show that the PS load is completed:

So Here is the final script in its entirety:

Now you can create your icon for your desktop so that you can easily access the script. I would save the script to your Scripts directory.

That will usually be C:\Users\’username’\Documents\WindowsPowerShell\Scripts or wherever directory you choose.

To start, right click the desktop and choose New > Shortcut
In the Target Field, enter the following for your PowerShell Shortcut, pointing to the path of your script:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -ExecutionPolicy Unrestricted -File “C:\Users\username\Documents\WindowsPowerShell\Scripts\ConnectO365All.ps1”

Click on the Advanced button and check the box: Run As Administrator
Under the General Tab, name your shortcut: (CompanyName) O365 All PowerShell
Click OK to save the shortcut to your desktop.

LAST BUT NOT LEAST, RUN THE FOLLOWING COMMAND BEFORE EXITING OR CLOSING YOUR POWERSHELL WINDOW. THIS WILL REMOVE ALL THE SESSIONS YOU’VE CONNECTED TO:

Get-PSSession | Remove-PSSession

HAPPY SCRIPTING!
LEARN, DO, LIVE!

References:
Connect to all O365 Services in one PowerShell Window
How to connect to all O365 Services through PowerShell
Connecting to Office 365 “Everything” via PowerShell

Checking Drive Space Volumes for DAG DB members through PowerShell

I had received a weird alert for a DB volume for a DAG member being below threshold. This was odd to me due to the fact that there were four DAG members and we only received an alert for one. I went into Azure Log Analytics and ran the following query to render a graph for the past 14 days showing the percent free space of the volume for all the DAG members.

Thanks Georges Moua for the query script!

Now the reason I can run the query this way is due to the fact that the Design of the DAG was correctly done and the DB folders are identical on all DAG members. The query rendered the following chart:

As you can see the Green DAG member is way below the other DAG members.

I next went to an Exchange Server in the DAG and got the volume data for all the members in the DAG:

EX02’s volume free space is far below the other DAG members

I went on EX02 and found that there was a subfolder named “Restore” that was not present on the other servers. I ran the following script to get the size of that folder in GB:

The folder size was 185 GB. Removing that folder, along with all subfolders/files, would balance the free space to the other DAG members. I ran the following cmdlet to remove the folder and all subfolders/files:

This remediated the alert and balanced the drive space across all DAG members.

POST YOUR COMMENTS OR QUESTIONS!
HAPPY TROUBLESHOOTING!