gcc11: dyld: Library not loaded: @rpath/libgfortran.5.dylib

[diekhans (Mark Diekhans)]

configure of R 4.1.0 fails with error:

:info:configure checking whether mixed C/Fortran code can be run... configure: WARNING: cannot run mixed C/Fortran code

config.log and main.log attached

[i0ntempest]

From CRAN website, under R arm64 download link:
"This release uses Xcode 12.4 and experimental GNU Fortran 11 arm64 fork. If you wish to compile R packages from sources, you may need to download GNU Fortran for arm64 from ​https://mac.R-project.org/libs-arm64."
My guess is MP doesn't have "experimental GNU Fortran 11 arm64" support yet.

[diekhans (Mark Diekhans)]

The previous version of the port of R (4.0.5 I think) did compile.

I looked at configure a bit, but didn't figure it out. It seems to want both gcc and clang and was trying to link a clang compile C file against and an FTN .o.

[ryandesign (Ryan Carsten Schmidt)]

The log shows:

ld: warning: dylib (/opt/local/lib/gcc11/libgfortran.dylib) was built for newer macOS version (11.3) than being linked (11.0)
dyld: Library not loaded: @rpath/libgfortran.5.dylib

So gcc11 must've been built with macOS 11.3 and MacPorts 2.6.4, a version which incorrectly set the MACOSX_DEPLOYMENT_TARGET to 11.3 when it should have set it to 11.0. This bug was fixed in MacPorts 2.7.0. gcc11 and any other affected ports should be rebuilt, i.e. (perhaps in a conditional that applies only on macOS 11) their revision should be increased and a check should be added to the portfile to ensure that MacPorts >= 2.7.0 is being used.

[ryandesign (Ryan Carsten Schmidt)]

Replying to i0ntempest:

My guess is MP doesn't have "experimental GNU Fortran 11 arm64" support yet.

MacPorts gcc11 does have experimental arm64 support; see ​https://github.com/macports/macports-ports/blob/master/lang/gcc11/Portfile#L43-L53

[diekhans (Mark Diekhans)]

rebuilding gcc11 from source with ports 2.7.1 does not fix the problem, I am still getting:

configure:26169: checking whether mixed C/Fortran code can be run
ld: warning: dylib (/opt/local/lib/gcc11/libgfortran.dylib) was built for newer macOS version (11.3) than being linked (11.0)
dyld: Library not loaded: @rpath/libgfortran.5.dylib
  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/work/R-4.1.0/./conftest
  Reason: image not found

[kencu (Ken)]

can you try rebuilding "libgcc11" from source?

sudo port -f uninstall libgcc11
sudo port -v -s install libgcc11

should do it.

MacPorts idiosyncracy.

[kencu (Ken)]

for reasons that make sense once you know the big picture of MacPorts, the gcc libraries are built and installed separately from the gcc port.

So when you rebuilt gcc11, you actually did not rebuild and install the library that is causing the issue. That library is installed by a separate port, libgcc11.

Even less obvious perhaps, the gcc 5 through gcc 10 libraries don't exist for the most part...those gcc versions all use the libraries from the current gcc version, which is libgcc11 at the moment.

[diekhans (Mark Diekhans)]

ah, thanks; I fooled by port provides ;-)

with rebuild libgcc11 from source, the "built for newer macOS version (11.3)" warning goes away, however the check for C linking with FORTRAN still fails. It sees odd that this test is run with clang-mp-12 if it wants to use gcc11.

I can try other suggestions; and can actually dig into debugging in a week or so.

configure:25995: /opt/local/bin/clang-mp-12 -o conftest -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch arm64 -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch arm64 conftest.c cfortran_test.o -ldl -lm   -L/opt/local/lib -L/opt/local/lib/gcc11/gcc/arm64-apple-darwin20/11.1.0 -L/opt/local/lib/gcc11/gcc/arm64-apple-darwin20/11.1.0/../../.. -lgfortran -lm >&5
configure:25995: $? = 0
configure:26037: result: lower case, underscore, no extra underscore
configure:26111: checking whether /opt/local/bin/gfortran-mp-11 appends underscores to external names
configure:26127: result: yes
configure:26139: checking whether /opt/local/bin/gfortran-mp-11 appends extra underscores to external names
configure:26155: result: no
configure:26169: checking whether mixed C/Fortran code can be run
dyld: Library not loaded: @rpath/libgfortran.5.dylib
  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/work/R-4.1.0/./conftest
  Reason: image not found
configure:26233: WARNING: cannot run mixed C/Fortran code
configure:26235: error: Maybe check LDFLAGS for paths to Fortran libraries?

[kencu (Ken)]

I'm trying to understand how that @rpath reference is getting in there to mess you up.

It's not finding the library due to that.

Is your library installed like mine, with a full path ref?:

$ otool -L  /opt/local/lib/libgcc/libgfortran.5.dylib
/opt/local/lib/libgcc/libgfortran.5.dylib:
	/opt/local/lib/libgcc/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/opt/local/lib/libgcc/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
	/opt/local/lib/libgcc/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

[diekhans (Mark Diekhans)]

no, it doesn't have @rpath expanded:

 % otool -L  /opt/local/lib/libgcc/libgfortran.5.dylib
objc[3151]: Class AMSupportURLConnectionDelegate is implemented in both /usr/lib/libauthinstall.dylib (0x1ff485160) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x114b142b8). One of the two will be used. Which one is undefined.
objc[3151]: Class AMSupportURLSession is implemented in both /usr/lib/libauthinstall.dylib (0x1ff4851b0) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x114b14308). One of the two will be used. Which one is undefined.
/opt/local/lib/libgcc/libgfortran.5.dylib:
	@rpath/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

[diekhans (Mark Diekhans)]

Looks like all of the libgcc libraries have @rpath rather than having it expanded

% otool -L  /opt/local/lib/libgcc/*.dylib
/opt/local/lib/libgcc/libatomic.1.dylib:
	@rpath/libatomic.1.dylib (compatibility version 4.0.0, current version 4.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libgcc_s.1.1.dylib:
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libgcc_s.2.dylib:
	/opt/local/lib/libgcc/libgcc_s.2.dylib (compatibility version 2.0.0, current version 2.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libgfortran.5.dylib:
	@rpath/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libgomp.1.dylib:
	@rpath/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libitm.1.dylib:
	@rpath/libitm.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libobjc-gnu.4.dylib:
	@rpath/libobjc-gnu.4.dylib (compatibility version 5.0.0, current version 5.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libssp.0.dylib:
	@rpath/libssp.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
/opt/local/lib/libgcc/libstdc++.6.dylib:
	@rpath/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.29.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

[kencu (Ken)]

Now we know what is wrong. Next is why and how to fix it.

I know about this issue with gcc, it's a planned change, but I thought this wasn't actually happening just now. Guess not.

We (and software builds) have not yet come to work right with @rpath'd libgcc libraries.

Chris, FYI.

[kencu (Ken)]

The branch we are using for the gcc11/libgcc11 build on arm is the one that has the @rpath enhancements added by iain.

So that is why this is being seen on arm.

Specifically this commit:

​https://github.com/fxcoudert/gcc/commit/d02ad2bd3a628d11230265e8cd511bf83344260f

[kencu (Ken)]

Well, most likely it will be required to rewrite all the libgcc11 libraries with full paths on installation, and probably tweak all the binaries too, if a more elegant procedure to configure gcc to turn @rpath off.

Even if gcc11 were tweaked to supply an additional @rpath reference to where the libgcc libs are by default located to all binaries it makes (do-able), all the other gcc versions used on MacPorts would not understand that. At the moment the Intel gcc versions aren't using @rpath, but the plan is to add it soon.

Right now, gcc10 says it is building on arm64, but not sure if it is really building usable software against real libraries, because it will use the libgcc11 libraries which have @rpath in them too.

For the moment, it seems gcc on arm64 must be challenged.

[ryandesign (Ryan Carsten Schmidt)]

Has any effort been made to report the difficulties that using @rpath is causing us to the developers of gcc to see what the best method would be to disable that new feature? If not, can someone do that?

[kencu (Ken)]

constant contact with Iains on this topic. Jeremy also involved, and others.

[kencu (Ken)]

the current troubles we are having with gcc11 on arm are not solely due to @rpath, however.

FX's fork has some differences from the main gcc11 it appears; there is a patch in the homebrew gcc11 formulae that looks related to this issue that we don't have (and perhaps don't want, but that takes some thinking).

I have not had an arm64 machine to work with for a while to fix these, but one has recently become available again.

[ryandesign (Ryan Carsten Schmidt)]

The revisions of gcc11 and libgcc11 were increased in [52cf3d239d60402dcd3843f927810e9b57a7c63f/macports-ports], resolving the deployment target issue (if they hadn't already been resolved before). Retitling the ticket since that isn't actually the cause of the problem.

[kamischi (Karl-Michael Schindler)]

The problem is also present on ortep3 on arm. I am also working on a new port named mopac, which builds the Fortran code with cmake and has the same problem. What do you think about a post build work around with install_name_tool as a preliminary fix for arm?

This is the preliminary fix from my pull request for ortep3:

post-build {
 # Preliminary fix of not finding the fortran libraries
     if { ${os.arch} eq "arm" } {
         set libgfortran [glob -tails -directory ${prefix}/lib/libgcc libgfortran.*.dylib]
         set libquadmath [glob -tails -directory ${prefix}/lib/libgcc libquadmath.*.dylib]
         system -W ${worksrcpath} "install_name_tool -change @rpath/${libgfortran} @executable_path/../lib/libgcc/${libgfortran} ortep3"
         system -W ${worksrcpath} "install_name_tool -change @rpath/${libquadmath} @executable_path/../lib/libgcc/${libquadmath} ortep3"
     }
 }

[cmstuber (Chris Stuberg)]

I'm on arm64 and seeing this issue, I believe, since the portfile change in the last handful of days. I'm pretty sure it was working last week for me but can't prove it. After uninstalling and re-installing of all my ports I get this running a simple gfortran11 test. I then tried building from source and got the same error, I also went and installed the previous commit and got the same error.

gcc12 is now installed (and working), which wasn't the case last week when I believe gcc11 was working properly.

dyld: Library not loaded: @rpath/libgfortran.5.dylib

macports_test # otool -L /opt/local/lib/libgcc/libgfortran.5.dylib   
/opt/local/lib/libgcc/libgfortran.5.dylib:
	@rpath/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	@rpath/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

Any work around or other insight would be extremely helpful.

[kencu (Ken)]

gcc11 is currently broken, as is gcc10 and possibly gcc-devel (on Intel).

Only gcc12, and gcc-devel (on arm64 only) currently work properly.

[kencu (Ken)]

see 65472 for more details

[cjones051073 (Chris Jones)]

Gcc11 (and gcc10) now have the same fixes as gcc12 so should be working ok.