gmp @6.3.0: tests fail when built with clang

[haberg-1 (Hans Åberg)]

GMP 'make check' fails when built with later versions of Clang, but passes when built with GCC, so that should be the build dependency. Tried on MacOS 14. See:
​https://gmplib.org/list-archives/gmp-bugs/2024-June/005505.html
​https://gmplib.org/list-archives/gmp-bugs/2024-July/005506.html

[kencu (Ken)]

If you download the current gmp software from here:

​https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz

decompress it and just build in on a current arm64 Mac system, outside of MacPorts, just with standard tools (Xcode and CLT installed, clang and standard tools used) you get a 100% pass:

========================================
   GNU MP 6.3.0: tests/test-suite.log
========================================

# TOTAL: 8
# PASS:  8
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

I realize, looking at the above message and <​https://gmplib.org/list-archives/gmp-bugs/2024-June/005494.html>, there is an alignment error with the x86_64 assembly code that now flags on MacOS as recent MacOS versions are intolerant of unaligned pointers. We have several MacPorts tickets about software trying to force unaligned pointers and Xcode clang / linker rejecting that.

The unaligned pointer issue seemed to be in one assembly file bdiv_q_1.asm. I noticed that file was forcing a low alignment of 8:

​https://github.com/gmp-mirror/gmp/blob/14fe69d7f56e00917e9fd9ab616afc798a1af6c1/mpn/x86_64/bdiv_q_1.asm#L137

I wondered if that might be the problem. I haven't as yet tried to fix it though.

So it might be premature to say clang is broken here.

[kencu (Ken)]

there is a newer version of gmp here, not yet released:

​https://gmplib.org/download/snapshot/gmp-next/gmp-6.3.0-20240515185115.tar.zst

The arm64 build test summary looks good there too (I included the cxx tests in this test run):

============================================================================
Testsuite summary for GNU MP 6.3.0
============================================================================
# TOTAL: 22
# PASS:  22
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0

I'll try the x64_64 run next.

[kencu (Ken)]

you can force an arm64 Mac into Intel mode like this:

gmp-6.3.0 % arch -arch x86_64 zsh

and from then on, the system thinks it's running on an Intel processor:

% ./configure  --enable-cxx  && make -j 10        
checking build system type... westmere-apple-darwin24.0.0
checking host system type... westmere-apple-darwin24.0.0
checking for a BSD-compatible install... /usr/bin/install -c
...

as per the ticket, there are certainly plenty of errors with the x86_64 build. This was the last bit that printed:

============================================================================
Testsuite summary for GNU MP 6.3.0
============================================================================
# TOTAL: 53
# PASS:  20
# SKIP:  1
# XFAIL: 0
# FAIL:  32
# XPASS: 0
# ERROR: 0
============================================================================
See tests/mpn/test-suite.log

disabling assembly on Intel fixes all of the errors (this was suggested by a gmp developer but I don't see that anyone ever tested it):

% ./configure  --enable-cxx  --disable-assembly && make -j 10

============================================================================
Testsuite summary for GNU MP 6.3.0
============================================================================
# TOTAL: 22
# PASS:  22
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0

So the error is in the way the handwritten gmp assembly files are being handled / compiled by clang. I suspect it is related to the ALIGN(8) above, so I'll see if changing that makes any difference. Here, we might be in tricky territory, however.

I haven't tried a gcc build as yet.

[kencu (Ken)]

I tried changing the ALIGN(8) to ALIGN(16) in two places in mpn/x86_64/bdiv_q_1.asm and that did nothing to improve the test errors.

There are a lot of assembly files, it turns out, that set ALIGN(8) -- perhaps others need fixing, or perhaps we are barking up the wrong tree altogether.

More to sort out.

You can't really set gmp to build with gcc14, as gmp is a build dependency of gcc14. <​https://ports.macports.org/port/gcc14/details/>. So really, this has to be properly sorted out by upstream gmp, and just saying "clang is hosed" or "Apple is silly" is not really going to be the solution.

For now -- I would personally lean towards disabling assembly for Intel builds of gmp until they sort it out.

MacPorts has lots of smart people around -- perhaps someone knows enough x86_64 assembly to see what gmp is doing to make these errors happen.

[kencu (Ken)]

Now unfortunately it is currently difficult to build with gcc in MacPorts (or HomeBrew I believe) targeting a non-native arch. The gcc compilers installed by either package manager only compile for the native arch, and current gcc cross-compilers have not been properly set up.

So there is no way for me to build gmp using gcc on this arm64 Mac as an Intel build to test it.

I would have to build a custom intel cross compiling gcc to do that -- which is actually not all that hard, just have to do it manually.

Let me see if any of my Intel machines can run a new enough MacOS to demonstrate the problem. They are generally running OpenCore now, but even with that, I don't think I can get to a new enough OS to show this issue.