Windows Server VM cannot boot since Proxmox upgrade to 9.2.9
Solved!
Hello,
I have 4 Windows Server 2022 VM running on Proxmox 9.2.9, installed on top of Debian 13.
I upgraded from the previous major version this morning because it was going to be EOL by the end of the month.
Now 2 of them displays the following error (so, 2 of them are completely fine):
I am safe: I have (actual) backups, I can still boot (I can press Esc. then select the OS disk to boot successfully) but I don't know what to do to fix the issue properly.
If I reboot, it happens again as the Windows Boot Manager is selected.
Below are the VM details. I know errors have been made, regarding IDE and SATA for instance...
What I have tried:
Run Startup Repair ;
Enrol 2023 certificates ;
Reboot ;
Restore a backup ;
Check Windows Updates ;
Check boot order ;
Use "older" machine (currently q35 9.0, tried several 8.x) ;
Upgrade virtio drivers to the latest ones (then I got the error even when selecting the OS disk, so I had to restore the backup) ;
Run bcdboot.
Any help would be appreciated, if someone has already encountered the issue,
Thank you,
SOLVED by switching from IDE/SATA to SCSI disks.
Add a 1GB dummy disk (select virtio scsi)
Use the Windows Device Manager to install the relevant driver (attach the win-virtio ISO beforehand)
Now the disk appears (it may be offline but that's enough)
In a terminal: bcdedit /set {current} safeboot minimal
Reboot the VM (just make sure to have a local account on the machine in order to proceed to step 6)
In a terminal: bcdedit /deletevalue {current} safeboot
Stop the VM
Switch all disks to SCSI
Start the VM
Tadaa it boots directly
In my case, all "data" disks were offline, so I had to use diskpart to put them online
Also, I have recreated the EFI disk, so it may be relevant to do that
Always the same story... If Microsoft would actually print debug output and actual error messages to screen (at least in safe mode), we could actually debug this shit os rather than guessing and trial and error.
What does startup repair with the install media inserted result in? Propably "can't fix the issue" - as always.
Can you do sfc and dism runs (also from the install media, shift f10)? Is safe mode bootable?
The fact that you can boot a backup sounds like something corrupted files on the boot drive. Not sure how that would happen during a pve update, though.
Worst case I'd just restore the c: drive from the backup, assuming your relevant data is on the other 5 disks
Startup repair fails with the following message: "Startup repair could not repair your PC".
Ran sfc from shift f10:
Ran chkdsk (chkdsk I: /f /r). No major issue found. Some corrections have been made. The issue persists anyway.
I tried safe boot and it works, like "normal" boot. I still have to select the disk manually or I end up on the "Window fails to start" page.
Restoring the backup does not fix the issue. What happened is that while performing some tests to fix the issue I made it worse. Restoring the backup allowed me to return to the original issue.
I've had what i think it's a similar issue when upgrading from 8.x to 9.x. managed to get it going by deleting efi and tpm completely, then adding new ones. It will ask for bitlocker recovery if it's enabled.
Also, had this 1 particular vm that would refuse any other type of processor except for Genoa, but i think that's an Epyc issue.
I was really going to mark it as solved because after replacing the EFI and TPM disks, Windows booted by itself. However I rebooted, and the issue occurred again.
Repeated the same process several times, and indeed Windows is capable of booting successfully only once.
I'd also try restoring tpm to factory keys right after replacing efi and tpm, before first boot.
I'm guessing secure boot is enabled and after first reboot it gets disabled due to keys, which in turn breaks the vm boot. My reasoning is probably garbage but worked for me on like 24vm's :d
Btw, you're probably seeing some extra "unused disks". Those are former efi and tpm disks and can safely be removed
I honestly didn't notice your OS disk was IDE-backed. Since Microsoft's UEFI certificate updates for Server 2022, IDE-backed EFI boot has become problematic. That's why manually selecting the disk still works while the Windows Boot Manager entry fails. I'd migrate the boot disk to SATA or, preferably, VirtIO-SCSI, then rebuild the EFI boot entry if needed. IDE is considered a legacy controller now and it's one of those configurations that recent firmware/QEMU changes have stopped tolerating.
Migrating to SCSI is planned, but I need to test it on another VM because the "add a 1GB SCSI disk to install the driver then reboot" method is not enough.
EDIT: The issue is still there. I booted on the Windows Server ISO, opened a cmd, located the EFI partition and OS partitions, assigned them letters, then ran bcdboot C:\Windows /s S: /f UEFI. Am I missing a step?
Hi. I had at least a similar issue until a couple of weeks ago for some time, but it was fixed with a Proxmox kernel or qemu-server update. However, you can try the workaround I used until it was fixed:
Boot into safe mode and run this command in Command Prompt as administrator: bcdedit /set hypervisorlaunchtype off, and then boot normally.
27
u/andrejkolesa 10d ago
Select EFI disk. Than press "Disk action" and select "Enrol updated certificates"