Opened 3 years ago

Closed 2 years ago

Last modified 2 years ago

#64146 closed enhancement (fixed)

rust on Mac OS 10.6.8?

Reported by: catap (Kirill A. Korinsky) Owned by: catap (Kirill A. Korinsky)
Priority: Normal Milestone:
Component: ports Version: 2.7.1
Keywords: Cc: mascguy (Christopher Nielsen), Wowfunhappy (Jonathan), evanmiller (Evan Miller), MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Port: rust

Description

Here a ticket that prevents my work to be forgotten.

I feel that with some future work rust can run on macOS 10.6.

I think that it is possible to support rust on macOS 10.6. It can use MacPorts' llvm/clang to compile that overstep mussed on this system thread local features.

Anyway, the question is how to run some rustc to build a nice one.

My current code is available here: https://github.com/catap/macports-ports/tree/rust-macOS-10.6 and it discovered that dynamicly linked thread local functions and variables can't work on 10.6 because dyld don't load it :)

The condition https://github.com/apple-open-source/macos/blob/10.6.8/dyld/src/ImageLoaderMachOCompressed.cpp#L567 prevents from it.

I have no idea how to overwritede this function which is statically linked inside dyld.

Anyway, this recent features and if try rust before it, anything before https://github.com/rust-lang/rust/commit/641d3b09f41b441f2c2618de32983ad3d13ea3f it can be load by 10.6.

Anyway, 1.53.0 is crashed as

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
0x00000001c528dbde in std::sys_common::thread_info::ThreadInfo::with::_$u7b$$u7b$closure$u7d$$u7d$::h126ae42a0a4323c8 ()
(gdb) bt
#0  0x00000001c528dbde in std::sys_common::thread_info::ThreadInfo::with::_$u7b$$u7b$closure$u7d$$u7d$::h126ae42a0a4323c8 ()
#1  0x00000001c528f197 in std::panicking::default_hook::hda3cfa32e0773aac ()
#2  0x00000001c528ff55 in std::panicking::rust_panic_with_hook::h1cdaa4b48d8ea4a1 ()
#3  0x00000001c528fa69 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h51157b4dbc4c5d91 ()
#4  0x00000001c528c0d8 in std::sys_common::backtrace::__rust_end_short_backtrace::h8f7fee8b4c5a0c05 ()
#5  0x00000001c528f9fa in rust_begin_unwind ()
#6  0x00000001c531dfaf in core::panicking::panic_fmt::h28a37220b8fae52a ()
#7  0x00000001c531df07 in core::panicking::panic::hc0de1ec470c25a9a ()
#8  0x00000001c528dde9 in std::sys_common::thread_info::set::hc916da4382f84dce ()
#9  0x00000001c5290399 in std::rt::lang_start_internal::h37583e1fa93097c4 ()
#10 0x00000001bd7f2bb9 in main ()
(gdb)

and additional digging shown that this is again thread local function which is shared accross libraries.

The next logical step is try to use static build of rustc. Static here means static linked with rust std and rustc-driver.

I can produce it by hand and test it, but if it work I still have no idea how to bootstrap it on 10.6, because if I hardcde rust 1.57.0 for example, in few releases 10.6 will be broken again.

:(

Attachments (1)

rust-10.6.log.bz2 (927.5 KB) - added by jmroot (Joshua Root) 2 years ago.

Download all attachments as: .zip

Change History (22)

comment:1 Changed 3 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added

comment:2 Changed 3 years ago by mascguy (Christopher Nielsen)

Port: rust added
Version: 2.7.1

comment:3 Changed 3 years ago by mascguy (Christopher Nielsen)

Owner: set to catap
Status: newassigned

comment:4 Changed 3 years ago by catap (Kirill A. Korinsky)

Ok, I've successfully build rustc for MACOSX_DEPLOYMENT_TARGET=10.6.

It was a tricky and I'd love to keep here how I did it.

The first things that I need is LLVM from this brnach: https://github.com/catap/llvm-project/tree/macOS-emulated-tlv it contains the master branch plus one commit. Which one? Enabling emulation of TLV before macOS 10.7. I've also reported this patch to upstream https://reviews.llvm.org/D115250.

From my point of view we may enable it branch for all clang/llvm versions on all platforms. Anyway, until it is done I need LLVM/clang from this branch.

I've configured this LLVM/clang as:

cmake -S llvm -B build -G Ninja \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=/Users/catap/src/llvm-project/dist \
      -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
      -DLLVM_TARGETS_TO_BUILD=X86 \
      -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt"

and build it as

cmake --build build

and finally install it as

cmake --install build

So, now I have a folder /Users/catap/src/llvm-project/dist which contains LLVM/Clang which supports _thread on macOS before 10.7 on my laptop under 12.

I've used rust from git repo with local root f1edd0429582dd29cccacaf50fd134b05593bd9c which is tag:1.57.0.

The first step is incorporate a patch which to LLVM inside rust, like this: ` cd src/llvm-project git remote add catap https://github.com/catap/llvm-project git remote update git cherry-pick catap/macOS-emulated-tlv cd ../../ git add src/llvm-project git commit -S -m 'Includes catap/llvm-project:macOS-emulated-tlv' `

I must create a commit to be protected from git submodule update which may dissmiss my cherry-pick ;)

Now, let's configure rust: ` env MACOSX_DEPLOYMENT_TARGET=10.6 \

PATH=/Users/catap/src/llvm-project/dist/bin:$PATH \ LD_LIBRARY_PATH=/Users/catap/src/llvm-project/dist/lib \ DYLD_LIBRARY_PATH=/Users/catap/src/llvm-project/dist/lib \ PKG_CONFIG_PATH=/Users/catap/src/llvm-project/pkgconfig CC=clang CXX=clang++ \ ./configure \ --default-linker=clang \ --build=x86_64-apple-darwin \ --enable-extended \ --tools=rustfmt,clippy,rls \ --set=target.x86_64-apple-darwin.linker=clang \ --set=target.x86_64-apple-darwin.ar=/Users/catap/src/llvm-project/dist/bin/llvm-ar \ --set=target.x86_64-apple-darwin.ranlib=/Users/catap/src/llvm-project/dist/bin/ranlib

` and build it: ` env MACOSX_DEPLOYMENT_TARGET=10.6 \

PATH=/Users/catap/src/llvm-project/dist/bin:$PATH \ LD_LIBRARY_PATH=/Users/catap/src/llvm-project/dist/lib \ DYLD_LIBRARY_PATH=/Users/catap/src/llvm-project/dist/lib \ PKG_CONFIG_PATH=/Users/catap/src/llvm-project/pkgconfig CC=clang CXX=clang++ \ make dist

`

after a while I'll have distributive which I can run on macOS 10.6.

I have no idea how to bootstrap rustc on the first place without repeating all this step inside upstream, and if anyone has an idea, I'd love to hear it.

Because right now it is chicken and the eggs sort of issue.

comment:5 Changed 3 years ago by kencu (Ken)

Indeed, you will need that llvm/clang patch to enable thread_local_storage using llvm using emulated tls; it was a real find to have such a straightforward patch to accomplish this on darwin a few years ago.

Upstream will never enable this hack on darwin < 10.7, but I applaud your gumption :> Upstream has officially killed all support for older darwin systems, and they have drawn the bar at about 10.14 now. The only reason it builds on anything earlier is because of all (my) our patches making it so.

I would have hoped we might apply pretty much all of our existing clang/llvm patches from llvm-13 to the llvm-13 that is used in rust, as those patches all have a role, have been used for years and years and are very well vetted. (I think llvm-13 is the one used in rust, if not then use the patches from our llvm that they do use.) It should be pretty trivial to do that instead of trying to keep up with yet another branch of llvm to be maintained for rust.

Also, don't forget there is the need for cxa_thread_atexit function to be provided in libc++abi.dylib, which we do in the libcxx port, where this is built. That function is provided there as that is where upstream provides it on all the systems that use emulated_tls, and I believe it is not supposed to be statically linked into each individual executable as it won't work properly that way (which would be the more trivial thing to do).

comment:6 Changed 3 years ago by catap (Kirill A. Korinsky)

Ken, do you think it is possible to enable emulation TLV for all llvm ports by default? :)

If yes, I'd love to create dedicated github repo and enjoy github actions to build rust which can be used for bootstrap rust on old platforms.

comment:7 Changed 3 years ago by Wowfunhappy (Jonathan)

Cc: Wowfunhappy added

comment:8 Changed 3 years ago by evanmiller (Evan Miller)

Cc: evanmiller added

comment:9 Changed 3 years ago by catap (Kirill A. Korinsky)

In 8799784ed7e6e20cd82987cf44479cd6db8aa265/macports-ports (master):

mrustc: update to 20220210

It also:

  • switched to rust 1.54.0;
  • blacklisted unsupported compilers;
  • added full rust and cargo bootstrap;
  • fixed build on macOS 10.6;
  • claim myself as maintainer.

See: #64146

comment:10 in reply to:  6 Changed 3 years ago by kencu (Ken)

Replying to catap:

Ken, do you think it is possible to enable emulation TLV for all llvm ports by default? :)

sure, possible. llvm/clang enables emulated_tls by default now on < 10.7 on MacPorts, and can be altered to use emulated_tls on newer systems quite easily.

Only issue at present is bootstrapping a fully-capable version of libcxx to support it.

If yes, I'd love to create dedicated github repo and enjoy github actions to build rust which can be used for bootstrap rust on old platforms.

sounds great

comment:11 Changed 3 years ago by lawlist

Is there any particular method of forcing rust to successfully install on OSX 10.6.8 at this point in time? I tried yesterday with sudo port install rust, but was met with a message stating there is a system requirement of 10.7 or greater.

comment:12 Changed 3 years ago by kencu (Ken)

almost.

Install the mrustc-rust port and you get rust 1.54 that mostly works (some thread local fixes still needed)

There is also an active PR to tweak this further

comment:13 Changed 3 years ago by evanmiller (Evan Miller)

To expand on @kencu's comment, the mrustc-rust port will install versions of rustc and cargo into /opt/local/libexec/mrustc-rust/bin. These generally cannot be used to build other Rust ports, but you're welcome to try them to build non-MacPorts projects.

I'd check back in a few weeks, as there is ongoing (daily) effort to bring "real" Rust to Snow Leopard, but it's not a small task.

comment:14 Changed 3 years ago by catap (Kirill A. Korinsky)

To be honest: I have moved much future and do have rust-1.58 locally, but I haven't yet pushed it because it requires significant time to build everything. Like a week and better to prepare everything and do it by build bot :)

Last edited 3 years ago by catap (Kirill A. Korinsky) (previous) (diff)

comment:15 in reply to:  11 Changed 3 years ago by catap (Kirill A. Korinsky)

@lawlist to be honest the only things which is blocked me to start pushing rust is https://github.com/macports/macports-ports/pull/13995

As soon as it delivered I'll move forward :)

comment:16 Changed 2 years ago by jmroot (Joshua Root)

In 05c69bf0ef5a184e5a963bb9517e778de7b26c2c/macports-ports (master):

rust: mark known_fail on 10.6 for now

Takes 6 hours to fail.

See: #64146

Changed 2 years ago by jmroot (Joshua Root)

Attachment: rust-10.6.log.bz2 added

comment:17 Changed 2 years ago by jmroot (Joshua Root)

Just attached the latest buildbot log to show the current state here.

comment:18 Changed 2 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Cc: MarcusCalhoun-Lopez added

comment:19 Changed 2 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

The problem seems to be that curl-sys looks for the static runtime library by running clang.
On Mac OS X 10.6, clang --print-search-dirs returns an empty directory.
Attempting to link with -L/lib/darwin -lclang_rt.osx results in a failure.

On my test machine, I was following the strategy from UsingTheRightCompiler, and clang failed instead of returning an empty directory.
MacPorts links the static library for other reasons Mac OS X 10.6, so the upstream error did not materialize.

There are pull request to fix the problem in MacPorts and upstream.

comment:20 Changed 2 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Resolution: fixed
Status: assignedclosed

In 97a9b7e1c919725137b080cfa3951bb6a5476fe9/macports-ports (master):

rust: find correct static runtime library

Fixes #64146

comment:21 Changed 2 years ago by catap (Kirill A. Korinsky)

In 38cf78bcc7849f98300f030e9af8942c3189c21b/macports-ports (master):

mrustc: update to 1.54-20220806

It also:

  • incorporated patches from rust-legacy-bootstrap.
  • switched to use legacysupport PG.
  • fixed an issue then mrustc might not build anything with thread local.

See: #64146

Note: See TracTickets for help on using tickets.