Opened 3 years ago

Closed 18 months ago

Last modified 18 months ago

#64442 closed defect (fixed)

bazel build points to builder directory _opt_bblocal_var_buildworker_ports_build_ports_devel_bazel

Reported by: essandess (Steve Smith) Owned by: missa-prime (Mohamed Issa)
Priority: Normal Milestone:
Component: ports Version: 2.7.1
Keywords: Cc: cjones051073 (Chris Jones), blair (Blair Zajac), mascguy (Christopher Nielsen)
Port: bazel, py-tensorflow-metadata

Description

I’m trying to build/update py-tensforflow-metadata and am hitting this (bizarre) bazel issue.

First, the build fails with the error:

:info:build Execution platform: @local_config_platform//:host
:info:build Use --sandbox_debug to see verbose messages from the sandbox
:info:build xcrun: error: can't exec '/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_devel_bazel/bazel-3.7/work/bazelwrap/cc' (errno=No such file or directory)

Weird that bazel wants to look in a MacPorts build directory.

Searching for this string in ${worksrcpath}, it appears in the file created during the build stage:

${worksrcpath}/bazel_build/install/<hash>/embedded_tools/tools/osx/crosstool/wrapped_clang.cc:
  if (binary_name == "wrapped_clang_pp") {
    tool_name = "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_devel_bazel/bazel-3.7/work/bazelwrap/cxx";
  } else if (binary_name == "wrapped_clang") {
    tool_name = "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_devel_bazel/bazel-3.7/work/bazelwrap/cc";

This is not the file from https://github.com/bazelbuild/bazel/blob/master/tools/osx/crosstool/wrapped_clang.cc

To be clear, I am not building bazel-3.7 here. I am building py38-tensorflow-metadata, which uses bazel-3.7. I’ve attached the log file. It was produced with a build (please see attached Portfile) that adds bazel.build_opts-append --sandbox_debug, not that I see that helping a diagnosis.

FWIW, I looked in the binary /opt/local/libexec/bazel-3.7/bin/bazel for the string _opt_bblocal_var_buildworker_ports_build_ports_devel_bazel but it’s not there, at least not in plain text. I have no idea how this path is being introduced at the build stage.

Attachments (2)

main.log.xz (6.9 KB) - added by essandess (Steve Smith) 3 years ago.
main.log
Portfile (3.0 KB) - added by essandess (Steve Smith) 3 years ago.
Portfile for updated py-tensorflow-metadata

Download all attachments as: .zip

Change History (15)

Changed 3 years ago by essandess (Steve Smith)

Attachment: main.log.xz added

main.log

Changed 3 years ago by essandess (Steve Smith)

Attachment: Portfile added

Portfile for updated py-tensorflow-metadata

comment:1 Changed 3 years ago by blair (Blair Zajac)

Cc: blair added

comment:2 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

Owner: set to missa-prime
Status: newassigned

The replacement of the unqualified "clang" and "clang++" in wrapped_clang.cc with their full paths is being performed by the bazel 1.0 portgroup. The bazel 1.0 portgroup also requests the use of the compiler_wrapper 1.0 portgroup, thus the full compiler paths are to temporary wrapper scripts that are only valid for the duration of the port build. The "bazel/bazel-3.7" part of the path you showed tells us this path was generated while building the bazel-3.7 subport of the bazel port; this path must be baked into the files installed by bazel-3.7 somewhere, though I also cannot find it.

Regardless what compiler path is baked into bazel-3.7, it is wrong that py-tensorflow-metadata is trying to use it. Ports must use the compiler MacPorts tells them to use. However, given all the other problems we already know about with bazel, it would not surprise me if bazel makes it impossible to use the right compiler. If that's the case, then bazel must not bake temporary compiler paths into itself; it should bake in paths to compilers that are guaranteed to exist, like /usr/bin/cc and /usr/bin/c++.

comment:3 Changed 3 years ago by cjones051073 (Chris Jones)

As far as I was aware, the path to the compilers used to build bazel where never expose outside that builds, to ports using bazel. If this is now happening then something has changed to cause this...

comment:4 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

The bazel executable contains embedded zip-compressed data. It's in there.

With the bazel-3.7 @3.7.2_1 binary for Catalina:

$ cd /tmp
$ unzip /opt/local/bin/bazel-3.7
$ grep -C 2 'binary_name == "wrapped_clang"' embedded_tools/tools/osx/crosstool/wrapped_clang.cc
  if (binary_name == "wrapped_clang_pp") {
    tool_name = "clang++";
  } else if (binary_name == "wrapped_clang") {
    tool_name = "clang";
  } else {

After rebuilding it from source:

$ unzip /opt/local/bin/bazel-3.7
$ grep -C 2 'binary_name == "wrapped_clang"' embedded_tools/tools/osx/crosstool/wrapped_clang.cc
  if (binary_name == "wrapped_clang_pp") {
    tool_name = "/opt/local/var/macports/build/_Users_rschmidt_macports_macports-ports-ryandesign-fork_devel_bazel/bazel-3.7/work/bazelwrap/cxx";
  } else if (binary_name == "wrapped_clang") {
    tool_name = "/opt/local/var/macports/build/_Users_rschmidt_macports_macports-ports-ryandesign-fork_devel_bazel/bazel-3.7/work/bazelwrap/cc";
  } else {

The bazel-3.7 binary for macOS 10.15 was produced in April 2021 while the binary for macOS 12 x86_64 was made in November 2021.

There was a flurry of changes to the bazel 1.0 portgroup, starting with [5d9765ddd7d0c8f16d25beeca7be37fd74c42f1d/macports-ports] in early May 2021, that introduced the code that modified wrapped_clang.cc.

Since these changes to the bazel 1.0 portgroup have an effect on the files that get installed with bazel-3.7 (and maybe other bazel versions), the revisions of the bazel ports should be increased, either now or better yet after you've made further changes to mitigate the problem.

Last edited 3 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:5 Changed 3 years ago by telotortium (Robert Irelan)

I've also encountered this as an end user.

I have

port installed | grep bazel
  bazel @4.2.2_0 (active)

and I'm trying to build Anki from source on MacOS following their instructions, but I'm receiving a similar error:

INFO: Analyzed target //qt:runanki (2 packages loaded, 262 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_bytedance/9966f08c7f44483c2c460b2db85aad34/external/rules_rust/util/process_wrapper/BUILD.bazel:3:10: Compiling util/process_wrapper/process_wrapper.cc [for host] failed: (Aborted): wrapped_clang failed: error executing command external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -g0 -O2 -DNDEBUG '-DNS_BLOCK_ASSERTIONS=1' ... (remaining 32 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
xcrun: error: can't exec '/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_devel_bazel/bazel/work/bazelwrap/cc' (errno=No such file or directory)
Error in child process '/usr/bin/xcrun'. 71
Target //qt:runanki failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /Users/bytedance/misc/build/anki/ts/graphs/BUILD.bazel:43:8 Bundling Javascript ts/graphs/index.ts [esbuild] failed: (Aborted): wrapped_clang failed: error executing command external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -g0 -O2 -DNDEBUG '-DNS_BLOCK_ASSERTIONS=1' ... (remaining 32 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 4.444s, Critical Path: 3.29s
INFO: 44 processes: 32 internal, 12 darwin-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

comment:6 Changed 2 years ago by essandess (Steve Smith)

This is also an issue on py*-tfx-bsl.

comment:7 in reply to:  4 Changed 2 years ago by essandess (Steve Smith)

The issue appears to be that these reinplace lines in the bazel PG,

https://github.com/macports/macports-ports/blob/17650ce5ce243e23518ef25d27f135e702cb9872/_resources/port1.0/group/bazel-1.0.tcl#L267-L268

are incompatible with these reinplace lines in the bazel Portfile,

https://github.com/macports/macports-ports/blob/57e385306ceed921f829e0aa7a3bb29f9948ec83/devel/bazel/Portfile#L279-L280

The fix appears to be to be just keeping around an original version of the file ./tools/osx/crosstool/wrapped_clang.cc and copying this back in a post-build phase, or in post-destroot.

I'd post a PR for this, but I can't get bazel to build locally. Chris, is this correct? Would you please post a fix?

Last edited 2 years ago by essandess (Steve Smith) (previous) (diff)

comment:8 Changed 2 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added

comment:9 Changed 2 years ago by cjones051073 (Chris Jones)

Sorry, I've kinda given up on bazel myself. far to much of a PITA to deal with any more...

comment:10 in reply to:  9 Changed 2 years ago by mascguy (Christopher Nielsen)

Replying to cjones051073:

Sorry, I've kinda given up on bazel myself. far to much of a PITA to deal with any more...

I'll try to help with this.

Last edited 2 years ago by mascguy (Christopher Nielsen) (previous) (diff)

comment:11 in reply to:  9 Changed 2 years ago by essandess (Steve Smith)

Replying to cjones051073:

Sorry, I've kinda given up on bazel myself. far to much of a PITA to deal with any more...

Sorry, understood. Unfortunately bazel is necessary to maintain tensorflow tools, so hopefully the number of PITA bazel interactions can be limited to just enough to get TF.

comment:12 Changed 18 months ago by essandess (Steve Smith)

Resolution: fixed
Status: assignedclosed

In d514b1a4d3aa3ae496f26ab5f2101486570e8c06/macports-ports (master):

PG bazel-1.0.tcl: Bugfix

comment:13 Changed 18 months ago by essandess (Steve Smith)

In 94a406552293645e2b775cf17441ae5e65115455/macports-ports (master):

bazel: Bugfix, Update to version 6.1.1

Note: See TracTickets for help on using tickets.