qemu @8.0.4: Hangs when used by lima

[MidnightRocket]

When using ​sysutils/lima qemu hangs when trying to boot a Debian based VM. This results in lima not able to successfully boot a Debian VM with qemu. Nothing is even written to the serial logs in $HOME/.lima/<INSTANCE>/serial.log . I suspect it has something to do with how Macports builds qemu, as using qemu binaries manually build from source works as intended.

Using lima binaries from ​github.com/lima-vm/lima/releases does not make a difference.

Steps to reproduce

• Use limactl to create a debian vm
• Make sure the vmType is qemu (set by default)
• Start the vm with limactl start $name
• Observe that limactl never reaches a successful boot

Current solutions

Current solution that I know of

• Using qemu @7.1.0 works as intended.
• Manually build qemu from source

My system:

macOS 13.5.1 22G90 x86_64
Xcode 14.3.1 14E300c

[MidnightRocket]

The problem persists with qemu @8.0.4 and lima @0.18.0

[mephaust (Paul Handly)]

I think I have a handle on the problem and how to work around it - still digging into the proper fix.

Qemu VMs (and thus, lima) depend on EDK2 UEFI images, and these VMs don't boot very far at all if the UEFI image file is stored with APFS "transparent" compression. Uncompressing the file seems to work around the problem - I'm still looking into the proper fix.

To address this, I think we either need to figure out why Qemu's RAW mode mishandles APFS-compressed files (not so transparent after all, eh?) or why the image file's being transparently compressed in the first place. Or both!

Repro

 qemu-system-x86_64 -accel hvf -cpu host,-pdpe1gb -m 4096 -drive if=pflash,format=raw,readonly=on,unit=0,file=/opt/local/share/qemu/edk2-x86_64-code.fd -drive if=pflash,format=raw,readonly=on,unit=1,file=/opt/local/share/qemu/edk2-i386-vars.fd

Workaround

# install a CLI tool to inspect and modify APFS compression
> sudo port install afscompress
# check to see if UEFI image is compressed
> /opt/local/bin/afscompress -vl /opt/local/share/qemu/edk2-x86_64-code.fd 
/opt/local/share/qemu/edk2-x86_64-code.fd:
File is HFS+/APFS compressed.
Compression type: ZLIB in resource fork (4)
File content type: dyn.ah62d4rv4ge80q3a
File size (uncompressed; reported size by Mac OS 10.6+ Finder): 3653632 bytes / 3.7 MB (megabytes, base-10)
File size (compressed): 1789952 bytes / 1.7 MiB
Compression savings: 51.0%
Number of extended attributes: 0
Total size of extended attribute data: 0 bytes
Uncompressed file size reported in compressed header: 3653632 bytes

# decompress file (if compressed above)
> afscompress -vd /opt/local/share/qemu/edk2-x86_64-code.fd

For Lima specifically, I think you can also work around this by forcing VMs to use legacy BIOS mode.

[elventear (Pepe Barbe)]

Replying to mephaust:

I think I have a handle on the problem and how to work around it - still digging into the proper fix. ...

Workaround

I was able to use this exact workaround and fix this issue that was plaguing me on macOS 14.4 / x86_64.

[elventear (Pepe Barbe)]

qemu was recently updated to 9.0.0 and the issue persists. The workaround is still valid.

[elventear (Pepe Barbe)]

Same issue under aarch64. This solved the problem:

sudo afscompress -vd /opt/local/share/qemu/edk2-aarch64-code.fd

[Tblue (Tilman Blumenbach)]

Replying to elventear:

Same issue under aarch64. This solved the problem:

sudo afscompress -vd /opt/local/share/qemu/edk2-aarch64-code.fd

I assume this will not persist after upgrading qemu, correct?

A somewhat radical workaround that should survive upgrades is to do the following (see comment:17:ticket:67731):

set hfscompression no in $prefix/etc/macports/macports.conf

Of course, this will affect all future port activations, and thus might not be an acceptable workaround for everybody, but it works fine for me: I think I'm fine with a bit more space being wasted if it means things don't break on port upgrades.

---

I wonder if we can ensure during qemu activation time that this file is not compressed, even if the user has hfcompression enabled in macports.conf?