Opened 3 years ago
Last modified 3 years ago
#63072 reopened defect
py39-numpy @1.20.3_0+gcc11+openblas fails to build on aarch64
Reported by: | thermalecology | Owned by: | michaelld (Michael Dickens) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.7.1 |
Keywords: | arm64 | Cc: | kencu (Ken), Guymer (Thomas Guymer), platipodium (Carsten Lemmen) |
Port: | py-numpy |
Description
Attachments (1)
Change History (18)
Changed 3 years ago by thermalecology
comment:1 Changed 3 years ago by thermalecology
Keywords: | numpy added; R removed |
---|---|
Port: | py39-numpy added; R-4.1.0 removed |
Summary: | R 4.1.0 fails on aarch64 → py39-numpy fails on aarch64 |
comment:2 Changed 3 years ago by jmroot (Joshua Root)
Keywords: | arm64 added; numpy removed |
---|---|
Owner: | set to michaelld |
Port: | py-numpy added; py39-numpy removed |
Status: | new → assigned |
Summary: | py39-numpy fails on aarch64 → py39-numpy @1.20.3_0+gcc11+openblas fails to build on aarch64 |
comment:3 Changed 3 years ago by michaelld (Michael Dickens)
comment:4 Changed 3 years ago by michaelld (Michael Dickens)
ah ... yes it looks like gcc11 is broken ... doesn't provide libgcc_s.1.1.dylib ...
comment:5 Changed 3 years ago by michaelld (Michael Dickens)
what works for me is using +gccdevel or +gfortran ... not +gcc11 ...
comment:6 Changed 3 years ago by michaelld (Michael Dickens)
Cc: | kencu added |
---|
adding Ken as I believe the issue here is with GCC11 rather than NumPy ... maybe he can shed some thought on the matter!
comment:7 Changed 3 years ago by reneeotten (Renee Otten)
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
gcc11
appears to build fine right now on aarch64
. So I'm assuming that this issue has resolved itself and will close this ticket; if the problem is still present feel free to reopen.
comment:8 Changed 3 years ago by Guymer (Thomas Guymer)
Whilst I have been able to get gcc11
to build (by using the patch in 63677) I have not been able to install py39-numpy
as I have been getting the same error. I uninstalled gcc11
(and all +gcc11
ports), I installed gcc-devel
instead and I still get the same error (after running port clean py39-numpy
).
For info:
sh-3.2# port installed | grep gcc fftw-3 @3.3.10_0+gccdevel (active) gcc-devel @12-20211127_0 (active) gcc_select @0.1_9 (active) libgcc-devel @12-20211127_0 (active) OpenBLAS @0.3.18_0+gccdevel+lapack+native (active)
For info:
sh-3.2# port select --summary | grep gcc gcc mp-gcc-devel mp-gcc-devel none
The failed build:
sh-3.2# grep error /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/main.log ... :info:build /opt/local/lib/gcc-devel/gcc/aarch64-apple-darwin21/12.0.0/include-fixed/math.h:68:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__." :info:build 68 | # error "Unsupported value of __FLT_EVAL_METHOD__." ...
Note that it clearly is attempting to build against gcc-devel
but it is coming across the same error. As an aside, I find it interesting that when I list the lib
folder it appears as if the libgcc
files have not been updated.
sh-3.2# ls -hl /opt/local/lib/libgcc/ total 2152 -rwxr-xr-x 1 root admin 70K 30 Nov 19:37 libatomic.1.dylib -rwxr-xr-x 1 root admin 85K 30 Nov 19:37 libgcc_s.1.1.dylib -rwxr-xr-x 1 root admin 1.0M 30 Nov 19:37 libgfortran.5.dylib -rwxr-xr-x 1 root admin 257K 30 Nov 19:37 libgomp.1.dylib -rwxr-xr-x 1 root admin 124K 30 Nov 19:37 libitm.1.dylib -rwxr-xr-x 1 root admin 113K 30 Nov 19:37 libobjc-gnu.4.dylib -rwxr-xr-x 1 root admin 34K 30 Nov 19:37 libssp.0.dylib -rwxr-xr-x 1 root admin 2.0M 30 Nov 19:37 libstdc++.6.dylib
Is it possible that gcc11
did not clean up after itself when it was uninstalled? Given that I was able to get gcc11
to install at the start, why didn't py39-numpy
then install? Surely I should not need to revert to gcc-devel
if I can get gcc11
to install.
comment:9 Changed 3 years ago by Guymer (Thomas Guymer)
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Using a fresh install of gcc11
(and removal of gcc-devel
), py39-numpy
still fails to install for me. The exact same error message is present in the log file:
sh-3.2# grep error /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/main.log ... :info:build /opt/local/lib/gcc-devel/gcc/aarch64-apple-darwin21/12.0.0/include-fixed/math.h:68:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__." :info:build 68 | # error "Unsupported value of __FLT_EVAL_METHOD__." ...
comment:10 Changed 3 years ago by kencu (Ken)
gcc is supposed to predefine FLT_EVAL_METHOD to some value. eg from brew's gcc11 on Intel, we see:
% gcc-11 -dM -E - < /dev/null | grep FLT_EVAL #define __FLT_EVAL_METHOD__ 0
Apple's math.h accepts a few values:
*/ #if __FLT_EVAL_METHOD__ == 0 typedef float float_t; typedef double double_t; #elif __FLT_EVAL_METHOD__ == 1 typedef double float_t; typedef double double_t; #elif __FLT_EVAL_METHOD__ == 2 || __FLT_EVAL_METHOD__ == -1 typedef long double float_t; typedef long double double_t; #else /* __FLT_EVAL_METHOD__ */ #error "Unsupported value of __FLT_EVAL_METHOD__." #endif /* __FLT_EVAL_METHOD__ */
so -- what does your gcc-mp-11 compiler show?
gcc-mp-11 -dM -E - < /dev/null | grep FLT_EVAL_METHOD
comment:11 Changed 3 years ago by Guymer (Thomas Guymer)
Thank you for responding, I appreciate it. This is what my 2021 MacBook Pro shows:
foo@bar ~ % gcc-mp-11 -dM -E - < /dev/null | grep FLT_EVAL_METHOD #define __FLT_EVAL_METHOD__ 0 #define __FLT_EVAL_METHOD_TS_18661_3__ 0 #define __FLT_EVAL_METHOD_C99__ 0
For info:
foo@bar ~ % port installed | grep gcc fftw-3 @3.3.10_0+gcc11 (active) gcc11 @11.2.0_1 (active) gcc_select @0.1_9 (active) libgcc @5.0_0 (active) libgcc11 @11.2.0_1 (active) OpenBLAS @0.3.19_0+gcc11+lapack+native (active) openmpi-gcc11 @4.1.1_0+fortran (active)
... and:
foo@bar ~ % port select --summary | grep gcc gcc mp-gcc11 mp-gcc11 none mpi openmpi-gcc11-fortran openmpi-gcc11-fortran none
Thanks
comment:12 Changed 3 years ago by kencu (Ken)
it’s hard to understand how this file is still there:
/opt/local/lib/gcc-devel/gcc/aarch64-apple-darwin21/12.0.0/include-fixed/math.h
when gcc-devel and libgcc-devel are both gone.
I’d be interested in why that is not the gcc11 version instead, and also be interested in what that file says in relation to FLT_EVAL_METHOD
, if anything.
comment:13 Changed 3 years ago by Guymer (Thomas Guymer)
Hi,
Given the whole gcc-devel
versus gcc11
thing I thought that I should double check that the log file wasn't from an old installation attempt. I just did:
sh-3.2# port sync sh-3.2# port clean py39-numpy sh-3.2# port install py39-numpy
It failed again (this time I won't trim the grep
output):
sh-3.2# grep error /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/main.log :debug:archivefetch Fetching archive failed: The requested URL returned error: 404 :debug:archivefetch Fetching archive failed: The requested URL returned error: 404 :debug:archivefetch Fetching archive failed: The requested URL returned error: 404 :info:build copying numpy/core/tests/test_indexerrors.py -> build/lib.macosx-12.0-arm64-3.9/numpy/core/tests :info:build CCompilerOpt.cc_test_flags[1013] : testing flags (-Werror) :info:build extra options: '-Werror' :info:build extra options: '-Werror' :info:build extra options: '-Werror' :info:build extra options: '-Werror' :info:build extra options: '-Werror' :info:build extra options: '-march=armv8.2-a+fp16 -Werror' :info:build extra options: '-march=armv8.2-a+fp16+fp16fml -Werror' :info:build CCompilerOpt.dist_test[581] : CCompilerOpt._dist_test_spawn[716] : Command (/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/compwrap/cc/opt/local/bin/gcc-mp-11 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/common -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/npymath -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.c -o /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/.tmp/tmp4vsk12ia/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.o -MMD -MF /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/.tmp/tmp4vsk12ia/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.o.d -march=armv8.2-a+fp16+fp16fml -Werror) failed with exit status 1 output -> :info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.c:13:35: error: implicit declaration of function 'vfmlal_low_u32'; did you mean 'vfmlal_low_f16'? [-Werror=implicit-function-declaration] :info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.c:13:35: error: incompatible type for argument 1 of 'vget_lane_f32' :info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.c:14:36: error: implicit declaration of function 'vfmlslq_high_u32'; did you mean 'vmlsl_high_u32'? [-Werror=implicit-function-declaration] :info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimdfhm.c:14:36: error: incompatible type for argument 1 of 'vgetq_lane_f32' :info:build cc1: all warnings being treated as errors :info:build extra options: '-march=armv8.2-a+dotprod -Werror' :info:build CCompilerOpt.dist_test[581] : CCompilerOpt._dist_test_spawn[716] : Command (/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/compwrap/cc/opt/local/bin/gcc-mp-11 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/common -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/npymath -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimddp.c -o /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/.tmp/tmp4vsk12ia/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimddp.o -MMD -MF /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/.tmp/tmp4vsk12ia/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/numpy-1.21.5/numpy/distutils/checks/cpu_asimddp.o.d -march=armv8.2-a+dotprod -Werror) failed with exit status 1 output -> :info:build <stdin>:29:2: error: instruction requires: dotprod :info:build <stdin>:41:2: error: instruction requires: dotprod :info:build CCompilerOpt._parse_policy_werror[2095] : compiler warnings are treated as errors :info:build extra options: '-Werror -march=armv8.2-a+fp16' :info:build /opt/local/lib/gcc11/gcc/arm64-apple-darwin21/11.2.0/include-fixed/math.h:68:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__." :info:build 68 | # error "Unsupported value of __FLT_EVAL_METHOD__." :info:build error: Command "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/work/compwrap/cc/opt/local/bin/gcc-mp-11 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/include/numpy -Ibuild/src.macosx-12.0-arm64-3.9/numpy/distutils/include -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/umath -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/common -Ibuild/src.macosx-12.0-arm64-3.9/numpy/core/src/npymath -c build/src.macosx-12.0-arm64-3.9/numpy/core/src/umath/_umath_tests.dispatch.asimdhp.c -o build/temp.macosx-12.0-arm64-3.9/build/src.macosx-12.0-arm64-3.9/numpy/core/src/umath/_umath_tests.dispatch.asimdhp.o -MMD -MF build/temp.macosx-12.0-arm64-3.9/build/src.macosx-12.0-arm64-3.9/numpy/core/src/umath/_umath_tests.dispatch.asimdhp.o.d -Werror -march=armv8.2-a+fp16" failed with exit status 1 :error:build Failed to build py39-numpy: command execution failed :error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py39-numpy/main.log for details.
I think that there are two sets of matches that are of note here:
- the
error: implicit declaration of function ...
ones; and - the
error "Unsupported value of __FLT_EVAL_METHOD__."
one.
If I list the /opt/local/lib
folder then I get:
sh-3.2# ls -hl /opt/local/lib | grep gcc drwxr-xr-x 36 root admin 1.1K 26 Dec 08:26 gcc11 drwxr-xr-x 10 root admin 320B 26 Dec 08:25 libgcc lrwxr-xr-x 1 root admin 24B 24 Dec 16:22 libstdc++.6.dylib -> libgcc/libstdc++.6.dylib drwxr-xr-x 40 root admin 1.3K 26 Dec 11:13 openmpi-gcc11
Which makes more sense. Clearly the log output from message comment:9 was an old build attempt, I am sorry for that - that was my mistake. In answer to your question about FLT_EVAL_METHOD
:
sh-3.2# grep -A 4 -B 4 FLT_EVAL_METHOD /opt/local/lib/gcc11/gcc/arm64-apple-darwin21/11.2.0/include-fixed/math.h * Floating point data types * ******************************************************************************/ /* Define float_t and double_t per C standard, ISO/IEC 9899:2011 7.12 2, taking advantage of GCC's __FLT_EVAL_METHOD__ (which a compiler may define anytime and GCC does) that shadows FLT_EVAL_METHOD (which a compiler must define only in float.h). */ #if __FLT_EVAL_METHOD__ == 0 typedef float float_t; typedef double double_t; #elif __FLT_EVAL_METHOD__ == 1 typedef double float_t; typedef double double_t; #elif __FLT_EVAL_METHOD__ == 2 || __FLT_EVAL_METHOD__ == -1 typedef long double float_t; typedef long double double_t; #else /* __FLT_EVAL_METHOD__ */ # error "Unsupported value of __FLT_EVAL_METHOD__." #endif /* __FLT_EVAL_METHOD__ */ #if defined(__GNUC__) # define HUGE_VAL __builtin_huge_val() # define HUGE_VALF __builtin_huge_valf()
Thanks
comment:14 Changed 3 years ago by Guymer (Thomas Guymer)
Cc: | Guymer added |
---|
comment:15 Changed 3 years ago by kencu (Ken)
So agreed that this port should not usually be deployed on macports with "-Werror" enabled (that is a dev thing) but sometimes it is found to be useful.
That said, this should not be happening, AFAIK:
extra options: '-march=armv8.2-a+dotprod -Werror'
all of these need to go:
error: implicit declaration of function ...
and then the big puzzle might be solved, if we get rid of this headache (by the above fixes?):
<stdin>:29:2: error: instruction requires: dotprod
then maybe math.h
will preprocess right and this will disappear error "Unsupported value of __FLT_EVAL_METHOD__."
as there seems to be absolutely no reason that __FLT_EVAL_METHOD__
that is defined by the compiler (hopefully still is defined by the compiler, at least, as you showed above) to "0" will not properly be handled by the code you showed.
A few things to straighten out perhaps...
comment:16 Changed 3 years ago by Guymer (Thomas Guymer)
Hi,
Is there any update on this bug? I tried the new MacPorts release of NumPy version 1.22.0 and it ran into the same issues. I also tried the Python 3.10 version and it too failed. Does anyone know why the automated build of the port ( shown here: https://build.macports.org/builders/ports-12_arm64-builder/builds/34602 ) appears to work but running port install py39-numpy
does not?
Thanks,
Tom
comment:17 Changed 3 years ago by platipodium (Carsten Lemmen)
Cc: | platipodium added |
---|
NumPy 1.21.1 seems to work for me on my ARM64 / M1 Mac:
I'll try with +gcc11 to see if that makes any difference.