I ran across an issue with my VM backups saying that they were failing validation and not backing up properly, even though each VM showed success when checking the logs. I was getting a specific error in the backup logs:
Backup files health check has been completed
Failed to perform backup file verification Error: Data error (cyclic redundancy check). Failed to read data from the file [B:\Backups\LDLNET Other Backup\LDLNET BackupD2019-01-12T001234.vbk]. Agent failed to process method {Signature.FullRecheckBackup}.
So, I did some research and found a little known tool that is used to manually validate the Veeam backup files, basically because it’s a tool usually executed only by the technical support staff. It is located in the following folder (Version 9.5.0.1922):
C:\Program Files\Veeam\Backup and Replication\Backup\Veeam.Backup.Validator.exe
Its main use case is to verify the consistency of a backup created with Veeam Backup & Replication. It’s NOT SureBackup, that does another kind of control by starting the VM from the backup file, and is for sure more reliable. But if you do not want to start a SureBackup activity, or if you only have a Standard license lacking SureBackup, this tool can be a good alternative, or you can use it to check a backup file after it has been moved or if you had a consistency problem on the storage holding those files.
Now since I had a specific file that was showing an error, I wanted to run the command against that file to validate the backup. Here is the command that I ran:
.\Veeam.Backup.Validator.exe /file:”B:\Backups\LDLNET Other Backup\LDLNET BackupD2019-01-12T001234.vbk”
It started running against the file and it did fail as it did in the log files from the backup process. Here is the error message:
Skipping VM ‘B:\Backups\LDLNET Other Backup\LDLNET BackupD2019-01-12T001234.vbk ‘: File “LDLNET-VM01-flat.vmdk” is corrupted. Data error (cyclic redundancy check).
Failed to read data from the file [B:\Backups\LDLNET Other Backup\LDLNET BackupD2019-01-12T001234.vbk ].
Now, when looking at the backup job, I found that the file listed was the original full backup that I had completed when I originally changed the job for all the new VMs that were now listed in the backup job. Since that was the case and I did not have any VMs that were in a bad state, I deleted all the backup files from my storage and started another full backup of the VMs in the job.
Using the CLI tool to manually validate the backup file was very helpful in this case as it would help me decide to clear out a backup that would not restore properly, even with the incremental backups since the base full file was corrupt.
References:
Veeam Backup Validator: check the consistency of your backup files
