#69138 closed defect (fixed)
rust @1.75.0 : Does not build on macOS 10.11.6 - missing _getentropy and _clock_gettime
Reported by: | snowflake (Dave Evans) | Owned by: | MarcusCalhoun-Lopez (Marcus Calhoun-Lopez) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.8.99 |
Keywords: | Cc: | mascguy (Christopher Nielsen), mqudsi (Mahmoud Al-Qudsi), Dave-Allured (Dave Allured), sambthompson (Sam Thompson) | |
Port: | rust |
Description
Here's the error:
:info:build Undefined symbols for architecture x86_64: :info:build "_getentropy", referenced from: :info:build std::sys::unix::rand::imp::fill_bytes::h1e6f730ecd7d963f in std-35a9205a1c5a5 e53.std.d632d37dc9be385b-cgu.13.rcgu.o :info:build "_clock_gettime", referenced from: :info:build std::sys::unix::time::Timespec::now::h767a5632f4dda4a4 (.llvm.219440299164370 3167) in std-35a9205a1c5a5e53.std.d632d37dc9be385b-cgu.11.rcgu.o :info:build ld: symbol(s) not found for architecture x86_64 :info:build clang: error: linker command failed with exit code 1 (use -v to see invocation) :info:build
I will also upload the last 100 lines of the log which show linker warnings. There are a lot of them but I don't think it is affecting the build.
Attachments (2)
Change History (28)
Changed 9 months ago by snowflake (Dave Evans)
comment:1 Changed 9 months ago by rmottola (Riccardo)
I have the same issue on 10.7 Several linker items, but also the same symbols missing. I bet it affects more systems, but currently 10.5 and 10.6 bail out on rust due to compile errors, so they don't get to linking.
Do we have an entropy fallback in LegacySupport?
:info:build ld: warning: cannot export hidden symbol ___udivti3 from /opt/local/libexec/llvm-16/lib/clang/16/lib/darwin/libclang_rt.osx.a(udivti3.c.o) :info:build ld: warning: cannot export hidden symbol ___umoddi3 from /opt/local/libexec/llvm-16/lib/clang/16/lib/darwin/libclang_rt.osx.a(umoddi3.c.o) :info:build ld: warning: cannot export hidden symbol ___umodsi3 from /opt/local/libexec/llvm-16/lib/clang/16/lib/darwin/libclang_rt.osx.a(umodsi3.c.o) :info:build ld: warning: cannot export hidden symbol ___umodti3 from /opt/local/libexec/llvm-16/lib/clang/16/lib/darwin/libclang_rt.osx.a(umodti3.c.o) :info:build Undefined symbols for architecture x86_64: :info:build "_getentropy", referenced from: :info:build std::sys::unix::rand::imp::fill_bytes::h1e6f730ecd7d963f in std-35a9205a1c5a5e53.std.d632d37dc9be385b-cgu.13.rcgu.o :info:build "_clock_gettime", referenced from: :info:build __ZN3std3sys4unix4time8Timespec3now17h767a5632f4dda4a4E.llvm.6669113470407685218 in std-35a9205a1c5a5e53.std.d632d37dc9be385b-cgu.11.rcgu.o
comment:2 Changed 9 months ago by mascguy (Christopher Nielsen)
Cc: | mascguy added |
---|
comment:3 Changed 9 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
It would seem we need to add more symbols in legacy-support so that rust-bootstrap finds them.
I am running experiments now.
Unfortunately, these ports take quite a while to build.
comment:4 follow-up: 9 Changed 8 months ago by mqudsi (Mahmoud Al-Qudsi)
FYI this was caused by upstream deprecating support for anything before macOS (née OS X) 10.12.
This commit is responsible for this particular issue: https://github.com/rust-lang/rust/commit/090e9de5708388a21d66adbec91c871a48ac355e
A compatibility shim for getentropy(3)
/_getentropy()
would be the ideal way to go, but a patch reverting that commit would also work.
comment:5 Changed 8 months ago by mqudsi (Mahmoud Al-Qudsi)
Cc: | mqudsi added |
---|
comment:6 Changed 7 months ago by rmottola (Riccardo)
I have this issue also on 10.6 32bit now...
I wonder if there is a decent way to have getentropy: that is a shim that does something equivalent?
comment:7 follow-up: 10 Changed 7 months ago by kencu (Ken)
How about we add it to legacysupport?
Oh, wait ...
comment:8 Changed 7 months ago by kencu (Ken)
Getting stuff to use the additional symbols in legacysupport takes a variable amount of work.
One way that might be needed here is to relink the binary against the libSystem.dylib from LegacySupport that re-exports the additional symbols that legacysupport provides, in addition to all the standard stuff in the system's libsystem.
All this has been done already, of course -- but things change, new versions of stuff comes out, blah blah, and things need to be looked at again to see that they still work right.
comment:9 Changed 7 months ago by rmottola (Riccardo)
Replying to mqudsi:
FYI this was caused by upstream deprecating support for anything before macOS (née OS X) 10.12.
This commit is responsible for this particular issue: https://github.com/rust-lang/rust/commit/090e9de5708388a21d66adbec91c871a48ac355e
A compatibility shim for
getentropy(3)
/_getentropy()
would be the ideal way to go, but a patch reverting that commit would also work.
oh, that is nasty... it hampers so many good systems... I tried applying the patch you mentioned reverse and compilation fails futher:
Undefined symbols for architecture i386: "_clock_gettime", referenced from: std::sys::unix::time::Timespec::now::h30810a3321b633c8 (.llvm.16531347933994784664) in std-0d102736d5b7d56a.std.5eedb8253ab94e49-cgu.12.rcgu.o ld: symbol(s) not found for architecture i386
Do we have that in legacy support?
comment:10 Changed 7 months ago by rmottola (Riccardo)
Replying to kencu:
How about we add it to legacysupport?
Oh, wait ...
oh indeed and at a glance it really looks taken from the code rust removed...
But there is no update to legacysupport yet? how is macports-legacy-support working?
comment:11 Changed 7 months ago by joostdekeijzer (joost de keijzer)
Cc: | joostdekeijzer added |
---|
comment:12 Changed 7 months ago by Dave-Allured (Dave Allured)
Cc: | Dave-Allured added |
---|
comment:13 Changed 7 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
There is a pull request that attempts to fix this problem.
comment:14 Changed 6 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
comment:15 follow-up: 17 Changed 6 months ago by sambthompson (Sam Thompson)
I might be doing something wrong, but this fix isn't working for me on 10.11.6. I have ensured the PR#79 changes that have landed in legacy-support-devel via PR 23300 are active on my MacPorts environment (by checking include header changes are in place in /opt/local/include and forcing a build from source with -s
, just to be sure).
Log from port upgrade -t rust
attached.
Changed 6 months ago by sambthompson (Sam Thompson)
install attempt on 10.11.6 after application of PR 79
comment:16 Changed 6 months ago by sambthompson (Sam Thompson)
Cc: | sambthompson added |
---|
comment:17 follow-up: 18 Changed 6 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Replying to sambthompson:
I might be doing something wrong, but this fix isn't working for me on 10.11.6. I have ensured the PR#79 changes that have landed in legacy-support-devel via PR 23300 are active on my MacPorts environment (by checking include header changes are in place in /opt/local/include and forcing a build from source with
-s
, just to be sure).Log from
port upgrade -t rust
attached.
Some of the fixes have not quite made it into released versions of the ports.
To be 100% "fixed," a few things have to happen.
For systems < 10.8, a bug has to be fixed in the legacy-support port.
Either install legacy-support-devel or wait until the next version of legacy-support.
Wait until rust is updated.
Sorry for the delay, but Rust is a bit tricky to maintain for systems < 10.12.
comment:18 Changed 6 months ago by sambthompson (Sam Thompson)
Replying to MarcusCalhoun-Lopez:
Sorry for the delay, but Rust is a bit tricky to maintain for systems < 10.12.
Not at all! Sorry, I didn't understand there was also a change required on the rust side; I was only using the latest legacy-support-devel. Will try again once the update PRs have gone through.
I really appreciate that MacPorts continues to support these legacy OS versions, despite the effort involved.
comment:19 Changed 6 months ago by Christopher Nielsen <mascguy@…>
comment:20 Changed 6 months ago by sambthompson (Sam Thompson)
Confirmed Rust 1.77.1 now builds on 10.11.6. Many, many thanks to Marcus and other testers for this; looks like this rust upgrade needed a lot of work, not just on legacy-support.
comment:21 Changed 5 months ago by mqudsi (Mahmoud Al-Qudsi)
Not sure if I should comment here or possibly open a different issue, but is there any way for the legacy-support shims to be visible to downstream users of rustc
linking against its standard library?
For example, the rand
crate assumes that since rustc
bumped its minimum supported macOS version to one with _getentropy()
and _clock_gettime()
, it can just call them. But that causes undefined symbol linker errors even when using a macports legacy-support-infused rust toolchain (rust/cargo 1.78.0 under 10.10) to build the project with the rand
dependency. As rand
is quite a foundational crate, this limits compatibility significantly (the last rand
version without the hard _getentropy()
dependency was 0.7.x which is not semver-compatible with the current 0.8.x rand releases and requires a lot of hacking to dependent projects to get them to run under 0.7.x).
Is there any way we can make the legacy support _getentroy()
and _clock_gettime()
shims transitively visible to its dependents?
The linker error:
error: linking with `cc` failed: exit status: 1 <snip> Undefined symbols for architecture x86_64: "_clock_gettime", referenced from: std::sys::pal::unix::time::Timespec::now::hf41285c15e0a285b (.llvm.4974563244208805110) in libstd-dea1dec033daebae.rlib(std-dea1dec033daebae.std.4e58b1c7d3f5f121-cgu.05.rcgu.o) "_getentropy", referenced from: std::sys::pal::unix::rand::imp::fill_bytes::h39617d8376b4a809 in libstd-dea1dec033daebae.rlib(std-dea1dec033daebae.std.4e58b1c7d3f5f121-cgu.14.rcgu.o)
As an fyi/aside, I tried to define my own _gentropy()
shim in rust (in the same project that uses rand
), but it doesn't work as the linker is searching for a dynamically exported symbol and is not considering static symbols exported by the same project (this kind of stuff used to work in C++ — or maybe it's actually a macOS-specific limitation that stops it from working?):
#[no_mangle] pub unsafe fn _getentropy(buf: *mut core::ffi::c_void, buflen: usize) -> core::ffi::c_int { use std::io::Read; let slice = std::slice::from_raw_parts_mut(buf as *mut u8, buflen); let mut rand = match std::fs::File::open("/dev/urandom") { Ok(file) => file, Err(_) => return -1, }; match rand.read_exact(slice) { Ok(_) => 0, _ => -1, } }
---
Edit: actually, looking at the linker error it seems that rand
is only calling into rust's std
and that's where the dynamic call to _getentropy()
and _clock_gettime()
is coming from, so does std
just need to be patched in the rust
port?
comment:22 Changed 5 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
The MacPorts ecosystem adds _getentroy()
and _clock_gettime()
via the rust PG.
For non-Macports projects, you have to add the appropriate linker flags manually so that the legacy-support library is found and used.
For example, adding -Wl,/opt/local/lib/libMacportsLegacySystem.B.dylib
to the linker flags.
Are you asking if -Wl,/opt/local/lib/libMacportsLegacySystem.B.dylib
can be automatically used as a linker flag?
comment:23 follow-up: 25 Changed 5 months ago by mqudsi (Mahmoud Al-Qudsi)
Thanks for that reference.
Are you asking if
-Wl,/opt/local/lib/libMacportsLegacySystem.B.dylib
can be automatically used as a linker flag?
Shouldn’t it? Imagine the legacy-support project doesn’t exist. One “perfect” patch (from a compatibility and results perspective, not from a maintenance or brittleness perspective) would be to patch the std library to revert the upstream removal of the legacy macOS support shims or replace the calls to the missing apis with patched equivalents. The result would be a rust toolchain that would build everything without any workarounds on behalf of the end user, including for non-MP rust projects. I would *expect* the same from the workaround that was actually used.
The drawback is obviously a runtime dependency on the legacy-support libs, but I don’t think that’s an unreasonable trade off. But does it break anything else?
comment:24 Changed 5 months ago by joostdekeijzer (joost de keijzer)
Cc: | joostdekeijzer removed |
---|
comment:25 Changed 5 months ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Replying to mqudsi:
Thanks for that reference.
Are you asking if
-Wl,/opt/local/lib/libMacportsLegacySystem.B.dylib
can be automatically used as a linker flag?Shouldn’t it? Imagine the legacy-support project doesn’t exist. One “perfect” patch (from a compatibility and results perspective, not from a maintenance or brittleness perspective) would be to patch the std library to revert the upstream removal of the legacy macOS support shims or replace the calls to the missing apis with patched equivalents. The result would be a rust toolchain that would build everything without any workarounds on behalf of the end user, including for non-MP rust projects. I would *expect* the same from the workaround that was actually used.
The drawback is obviously a runtime dependency on the legacy-support libs, but I don’t think that’s an unreasonable trade off. But does it break anything else?
It may be possible and desirable to have -Wl,${prefix}/lib/libMacportsLegacySystem.B.dylib
automatically added.
I am afraid I would have to research this a bit.
comment:26 Changed 4 months ago by mqudsi (Mahmoud Al-Qudsi)
After playing around with this some and trying it out against real-world projects, I have unfortunately come to realize it might actually be necessary to patch the toolchain to apply this automatically to work around deficiencies in cargo's current abilities to specify linker flags when building transitive dependencies that include proc macros or build scripts (but only specifically where building for the host (i.e. sans --target
) or where building with --target x86_64-apple-darwin
).
The problem is that Cargo intentionally does not pass through RUSTFLAGS to build scripts and proc macros (directly or in your transitive dependencies) if you build with an explicit --target x86_64-apple-darwin
(or any other target), *and* there is no known workaround to pass through linker flags to the build scripts in this case. So for any rust project that has a *-sys
crate in its (transitive) dependencies, the link stage will fail due to missing symbol errors and you can't do anything about it.
I bugged it upstream and there's an active discussion going on, but it's tangential to a long-standing known issue with no resolution in sight.
The only "solution" I found was to hack the build scripts to remove explicit --target
from cargo calls then patch the install phase to copy artifacts out of $BUILD_DIR/cargo/build/{debug,release}
instead of $BUILD_DIR/cargo/build/x86_64-apple-darwin/{debug,release}
, but that's not really a workable solution.
(I'm thinking in terms of solutions that would work to provide macports ports for projects written in rust that explicitly specify --target
via their non-cargo build system on a target that requires LegacySupport.)
Last 100 lines of the log.