ProvisioningTaskError - Image Preparation failed

Terminating Error:
Image Preparation Finalization Failed
Stack Trace:
at Citrix.Orchestration.Base.PowerShellSdk.ProvisioningSchemeService.BackgroundTasks.ProvisioningSchemeTask.ThrowOnTerminatingError(SdkProvisioningSchemeAction sdkProvisioningSchemeAction)
at Citrix.Orchestration.Base.PowerShellSdk.ProvisioningSchemeService.BackgroundTasks.ProvisioningSchemeTask.WaitForProvisioningSchemeActionCompletion(Guid taskId, Action`1 actionResultsObtained)
at Citrix.Orchestration.Base.PowerShellSdk.ProvisioningSchemeService.BackgroundTasks.PublishProvisioningSchemeImageTask.RunTask()
at Citrix.Orchestration.Base.PowerShellSdk.BackgroundTaskService.BackgroundTask.Task.Run()

DesktopStudio_ErrorId : ProvisioningTaskError
ErrorCategory : NotSpecified
ErrorID : ImagePreparationFinalizationFailed
TaskErrorInformation : Terminated
InternalErrorMessage : Image Preparation failed

Environment

Hypervisor - Windows 2016 HyperVisor with SCVMM 2016 Rollup Update 7 (Supports Windows 2019 Guests)

XenDesktop - Build 1811 (Supports Windows 2019 Guests)

Guest O/S - Windows Server 2019 Standard Edition Build 1809

VDA Version - Build 1811

 

When attempting to create a new machine catalog, after 20 minutes, the catalog creation process fails with 'Preparation Finalization Failed'. This occurred duing the image preparation phase, after the base disk had been successfully copied.

I've been using the same infrastructure successfully for Windows 2016 XenApp VDAs without issue.

Citrix provides a couple of technotes on Image creation:

Updating a Catalog Fails During Image Preparation

Image Preparation Overview and Fault-Finding on Machine Creation Service

And a blog:

Machine Creation Service: Image Preparation Overview and Fault-Finding

To summarise, the snapshot of the master image should be copied, then a preparation VM is started to create the instruction disk. The preparation VM is shutdown and deleted, and the catalog is created / updated. If the image preparation process does not complete within 20 minutes, then the preparation VM is shut down and the catalog create / update process fails.

The articles include a few things to check, and suggest enabling logging of the image preparation process by adding a registry value

HKLM\Software\Citrix\MachineIdentityServiceAgent\LOGGING = DWORD 00000001

to create a log file (C:\image-prep.log) - here's mine

------------05 June 2019 00:36:24---------------
Started Image Prep, arg count 20
Read arguments, OS Rearm True, Office Rearm True, Third Party True, RequirePvd False, RequireWbc False, BuildPvd False, EnableDHCP True, AppScanning False, AppScanResultsFilePath
Reporting status to 'C:\Program Files\Citrix\PvsVm\Service\PersistedData\PreparationStatus.xml'.
No prior existence of status file detected. Clean prep execution.
Requesting start of winmgmt service
WMI ready.
Third party actions found
Office rearm tool locations are 2010:C:\Program Files\Common Files\microsoft shared\OfficeSoftwareProtectionPlatform\OSPPREARM.EXE and C:\Program Files (x86)\Common Files\microsoft shared\OfficeSoftwareProtectionPlatform\OSPPREARM.EXE, 2013:C:\Program Files\Microsoft Office\Office15\OSPPREARM.EXE and C:\Program Files (x86)\Microsoft Office\Office15\OSPPREARM.EXE, , 2016:C:\Program Files\Microsoft Office\Office16\OSPPREARM.EXE and C:\Program Files (x86)\Microsoft Office\Office16\OSPPREARM.EXE
Office rearm tool is not present, nothing to do
Enabling DHCP on all network adapters
Netsh tool completed, exit code 0
Netsh executed command 'int ipv6 show addresses' successfully
ReadIpv6Addresses: Found 0 addresses.
EnableDHCP: Checking adapter, Microsoft Hyper-V Network Adapter #2
EnableDHCP: Checking configuration, Microsoft Hyper-V Network Adapter
DHCP already enabled for Microsoft Hyper-V Network Adapter #2, nothing to do
Netsh tool completed, exit code 1
DUID = 00-01-00-01-24-86-fe-3f-00-15-5d-06-0a-12
status = 0
Found 10 interfaces
FreeMibTable status = 1
There are 1 interesting interfaces
Deleting keys 0001
Deleting DUID
Deleted keys
Trying to stop NetSetupSvc
Waiting for NetSetupSvc to stop
Timeout waiting for NetSetupSvc to stop
The OS already restarted NetSetupSvc
Waiting for NetSetupSvc to stop after we started it

The image preparation service is waiting for the Network Setup Service to stop.

https://docs.microsoft.com/en-us/windows-server/security/windows-services/security-guidelines-for-disabling-system-services-in-windows-server

The StartType is actually Manual (Triggered). If you open 'Control Panel - Network Connections' you'll see the service start. Close 'Network Connections' and about three minutes later, the service will stop.

The obvious solution to this problem was to stop the 'Network Setup Service' manually by logging onto the preparation VM using the Hypervisor console. At which point the preparation VM immediately shut down and the catalog creation process completed successfully. Great.

However I don't want to have to go through this on every vDisk update.

What if my VMs are in Azure? During the image preparation process, the NIC is disconnected so I can't RDP to the preparation VM.

If your hypervisor team is a 3rd party, they might not allow you access to the VMWare or Hyper-V console. Good luck troubleshooting in that case.

After digging around for registry tweaks for service idle timeouts for the NetSetupSvc (unsuccessfully), I decided on a scheduled task in the master image that would:

  • Start a Powershell script to check if the VM is in 'Private Mode'
  • Wait ten minutes
  • Then stop the Network Setup Service

Here's the script - rename to .ps1 and copy to C:\Windows

And here's the Scheduled Task configuration

Program: Powershell.exe

Argument: -ExecutionPolicy ByPass -NoProfile -File C:\Windows\Kill-netsetupsvc.ps1

The image update process now takes ten minutes instead of just a few minutes however it completes successfully and you don't have to watch the update and the preparation VM.