OpenBLAS @0.3.25 does not build universal arm64/x86_64

[kencu (Ken)]

/opt/local/var/macports/build/_opt_macportsofficial_macports-ports_math_OpenBLAS/OpenBLAS/work/compwrap/fc/opt/local/bin/gfortran-mp-12 -m64 -Os -O3 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64 -fPIC -msse3 -mssse3 -msse4.1 -c -o sgbtf2.o sgbtf2.f
/opt/local/var/macports/build/_opt_macportsofficial_macports-ports_math_OpenBLAS/OpenBLAS/work/compwrap/fc/opt/local/bin/gfortran-mp-12 -m64 -Os -O3 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64 -fPIC -msse3 -mssse3 -msse4.1 -c -o sgbtrf.o sgbtrf.f
gfortran-mp-12: error: unrecognized command-line option '-m128bit-long-double'
gfortran-mp-12: error: unrecognized command-line option '-m128bit-long-double'
gfortran-mp-12: error: unrecognized command-line option '-msse3'
gfortran-mp-12: error: unrecognized command-line option '-msse3'
gfortran-mp-12: error: unrecognized command-line option '-mssse3'
gfortran-mp-12: error: unrecognized command-line option '-mssse3'
gfortran-mp-12: error: unrecognized command-line option '-m128bit-long-double'
gfortran-mp-12: error: unrecognized command-line option '-m128bit-long-double'
gfortran-mp-12: error: unrecognized command-line option '-m128bit-long-double'
gfortran-mp-12: error: unrecognized command-line option '-msse3'
gfortran-mp-12: error: unrecognized command-line option '-msse3'
gfortran-mp-12: error: unrecognized command-line option '-msse3'
gfortran-mp-12: error: unrecognized command-line option '-msse4.1'
gfortran-mp-12: error: unrecognized command-line option '-msse4.1'
gfortran-mp-12: error: unrecognized command-line option '-mssse3'
gfortran-mp-12: error: unrecognized command-line option '-mssse3'
gfortran-mp-12: error: unrecognized command-line option '-mssse3'
make[2]: *** [sgetrf2.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [spotrf2.o] Error 1

looks like this is because our gcc can't build universal.

[kencu (Ken)]

having a universal openblas is needed to allow x86_64-only ports that use openblas to build on arm macs.

[catap (Kirill A. Korinsky)]

Ken, I think the root cause is that gfortran-mp-12 can't build anything for x86_64 on arm.

catap@Kirills-mini-m1 /tmp % port -v installed gcc12
The following ports are currently installed:
  gcc12 @12.3.0_0+stdlib_flag+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2023-06-17T02:13:23+0200'
catap@Kirills-mini-m1 /tmp % cat test.f90           
program hello
  print *, 'Hello, World!'
end program hello
catap@Kirills-mini-m1 /tmp % gfortran-mp-12 -arch arm64 -o test test.f90 
catap@Kirills-mini-m1 /tmp % file test 
test: Mach-O 64-bit executable arm64
catap@Kirills-mini-m1 /tmp % rm -f test                                 
catap@Kirills-mini-m1 /tmp % gfortran-mp-12 -arch x86_64 -o test test.f90
gfortran-mp-12: warning: this compiler does not support x86 ('-arch' option ignored)
catap@Kirills-mini-m1 /tmp % file test
test: Mach-O 64-bit executable arm64
catap@Kirills-mini-m1 /tmp % 

[kencu (Ken)]

Yes, until someone finally sorts out "driverdriver.c" you would have to set the build up to call the x86_64 gfortran compiler directly (ie not via the main gfortran-mp-12 file but via the direct x86_64_gfortran-mp-12 compiler) using the muniversal portgroup and some direct calls like you did in gcc10-bootstrap to build universal.

[kencu (Ken)]

well, you did it slightly differently than what I described in gcc10-bootstrap, using the "arch" command to drive the main driver, but achieved the same idea of an arch-specific gcc driver for each selected arch:

    foreach arch \$\{universal_archs_supported\} {
        lappend merger_configure_env(\$\{arch\}) \
                        \"CC=arch -arch \$\{arch\} \$\{prefix\}/libexec/${name}/bin/gcc\"
        lappend merger_configure_env(\$\{arch\}) \
                        \"CXX=arch -arch \$\{arch\} \$\{prefix\}/libexec/${name}/bin/g++\"
    }

[kencu (Ken)]

Oh oh -- it looks like our universal build of gcc12 arm64/x86_64 is not actually universal at all.

At least, as far as I can see, the arm64 build of gcc12 has nothing x86_64 in it:

% port -v installed | grep gcc
  gcc12 @12.3.0_0+stdlib_flag+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2023-06-17T08:32:19-0700'
  gcc12-libcxx @12.3.0_0+clang14 (active) requested_variants='' platform='darwin 22' archs='noarch' date='2023-06-17T07:46:36-0700'
  gcc_select @0.1_10 (active) requested_variants='' platform='darwin 22' archs='noarch' date='2023-06-17T07:46:37-0700'
  libgcc @6.0_0+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2023-06-17T08:11:23-0700'
  libgcc12 @12.3.0_0+stdlib_flag+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2023-06-17T08:11:14-0700'

% port contents gcc12 | grep x86_64 

% port contents gcc12 | grep bin/ | xargs file
/opt/local/bin/arm64-apple-darwin22-c++-mp-12:        Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-g++-mp-12:        Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gcc-12.3.0:       Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gcc-ar-mp-12:     Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gcc-mp-12:        Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gcc-nm-mp-12:     Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gcc-ranlib-mp-12: Mach-O 64-bit executable arm64
/opt/local/bin/arm64-apple-darwin22-gfortran-mp-12:   Mach-O 64-bit executable arm64
/opt/local/bin/c++-mp-12:                             Mach-O 64-bit executable arm64
/opt/local/bin/cpp-mp-12:                             Mach-O 64-bit executable arm64
/opt/local/bin/g++-mp-12:                             Mach-O 64-bit executable arm64
/opt/local/bin/gcc-ar-mp-12:                          Mach-O 64-bit executable arm64
/opt/local/bin/gcc-mp-12:                             Mach-O 64-bit executable arm64
/opt/local/bin/gcc-nm-mp-12:                          Mach-O 64-bit executable arm64
/opt/local/bin/gcc-ranlib-mp-12:                      Mach-O 64-bit executable arm64
/opt/local/bin/gcov-dump-mp-12:                       Mach-O 64-bit executable arm64
/opt/local/bin/gcov-mp-12:                            Mach-O 64-bit executable arm64
/opt/local/bin/gcov-tool-mp-12:                       Mach-O 64-bit executable arm64
/opt/local/bin/gfortran-mp-12:                        Mach-O 64-bit executable arm64
/opt/local/bin/lto-dump-mp-12:                        Mach-O 64-bit executable arm64

and neither is libgcc12:

% port contents libgcc12 | grep dylib | xargs file
/opt/local/lib/libgcc/libatomic.1.dylib:   Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libatomic.dylib:     Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgcc_s.1.1.dylib:  Mach-O universal binary with 1 architecture: [arm64:Mach-O 64-bit dynamically linked shared library arm64]
/opt/local/lib/libgcc/libgcc_s.1.1.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgcc_s.dylib:      Mach-O universal binary with 1 architecture: [arm64:Mach-O 64-bit dynamically linked shared library arm64]
/opt/local/lib/libgcc/libgcc_s.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgfortran.5.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgfortran.dylib:   Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgomp.1.dylib:     Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libgomp.dylib:       Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libitm.1.dylib:      Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libitm.dylib:        Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libobjc-gnu.4.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libobjc-gnu.dylib:   Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libquadmath.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libquadmath.dylib:   Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libssp.0.dylib:      Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libssp.dylib:        Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libstdc++.6.dylib:   Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libgcc/libstdc++.dylib:     Mach-O 64-bit dynamically linked shared library arm64
/opt/local/lib/libstdc++.6.dylib:          Mach-O 64-bit dynamically linked shared library arm64

so someone has some work to do.

[barracuda156]

Replying to kencu:

Oh oh -- it looks like our universal build of gcc12 arm64/x86_64 is not actually universal at all.

I think all GCC “universal” builds are not universal in a real sense, aside of gcc10-bootstrap. As we have seen recently in other ticket, gfortran does not work for i386 when gcc12 is built as +universal.

[barracuda156]

16. S. Why on earth it tries to use -m128bit-long-double? It’s a PowerPC flag used on ELF, AFAIK.

[catap (Kirill A. Korinsky)]

Sergey, I have no idea, but you may ask the upstream: ​https://github.com/xianyi/OpenBLAS/blob/develop/Makefile.system#L541-L556

[kencu (Ken)]

Replying to barracuda156:

Replying to kencu:

Oh oh -- it looks like our universal build of gcc12 arm64/x86_64 is not actually universal at all.

I think all GCC “universal” builds are not universal in a real sense, aside of gcc10-bootstrap.

this was by design. If universal is i386/x86_64, which it was for 15 years, gcc’s multilib works fine without gcc being fully universal.

gcc10-bootstrap is not built universal correctly either, by the way.

but for arm64/x86_64 we need a cross-arch universal….which I actually thought we had last I looked, but I guess not.

As we have seen recently in other ticket, gfortran does not work for i386 when gcc12 is built as +universal.

It always did before. That ticket is something new that is probably Ian’s path bug to fix, as per the other ticket. It’s a totally different kind of issue than the one here, so let’s not conflate.

This gcc issue is a MacPorts build problem.

[kencu (Ken)]

we’ll track the gcc issue in #67652

[stefanoaz (Steve R)]

OpenBLAS fails to build on Mac OS 14.1, still due to it not accommodating aarm64 (see below). It is not clear from this thread, where the core issue lies. There are many packages with openblas as a dependency, like octave and gnuradio.

:info:configure Command failed: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/build-x86_64" && /opt/local/bin/cmake -G "CodeBlocks - Unix Makefiles" -G "CodeBlocks - Unix Makefiles" -DCMAKE_BUILD_TYPE=MacPorts -DCMAKE_INSTALL_PREFIX="/opt/local" -DCMAKE_INSTALL_NAME_DIR="/opt/local/lib" -DCMAKE_SYSTEM_PREFIX_PATH="/opt/local;/usr" -DCMAKE_C_COMPILER="$CC" -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_OBJC_COMPILER="$CC" -DCMAKE_OBJCXX_COMPILER="$CXX" -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_POLICY_DEFAULT_CMP0060=NEW -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_COLOR_MAKEFILE=ON -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_MAKE_PROGRAM=/usr/bin/make -DCMAKE_MODULE_PATH="/opt/local/share/cmake/Modules" -DCMAKE_PREFIX_PATH="/opt/local/share/cmake/Modules" -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF -DCMAKE_INSTALL_RPATH="/opt/local/lib" -Wno-dev -DCOMMON_PROF=-pg -DNUM_THREADS=56 -DNO_AVX=ON -DNO_AVX2=ON -DNO_AVX512=ON -DCMAKE_AR=/opt/local/bin/ar -DCMAKE_NM=/opt/local/bin/nm -DCMAKE_OBJDUMP=/opt/local/bin/objdump -DCMAKE_RANLIB=/opt/local/bin/ranlib -DCMAKE_STRIP=/opt/local/bin/strip -DCMAKE_LINKER=/opt/local/bin/ld -DBUILD_SHARED_LIBS=ON -DCMAKE_OSX_DEPLOYMENT_TARGET="14.0" -DCMAKE_OSX_SYSROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk" -DCMAKE_OSX_DEPLOYMENT_TARGET="14.0" -DCMAKE_OSX_SYSROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk" -DCMAKE_OSX_ARCHITECTURES=x86_64 -DCMAKE_OSX_ARCHITECTURES=x86_64 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/OpenBLAS-0.3.25-x86_64 :info:configure Exit code: 1 :error:configure Failed to configure OpenBLAS

[kencu (Ken)]

and even after the transition to cmake, still won't build universal.

Different error, though. new log attached.

[kencu (Ken)]

openblas failure universal M1

[catap (Kirill A. Korinsky)]

Let me extract error from the log:

:info:configure CMake Error at cmake/utils.cmake:23 (file):
5772	:info:configure   file STRINGS file
5773	:info:configure   "/opt/local/var/macports/build/_opt_macportsofficial_macports-ports_math_OpenBLAS/OpenBLAS/work/OpenBLAS-0.3.25-x86_64/kernel/arm64/KERNEL.NEHALEM"
5774	:info:configure   cannot be read.
5775