Opened 4 years ago
Closed 17 months ago
#62570 closed defect (fixed)
py-tensorflow: pip whl is not a supported wheel on this platform.
Reported by: | essandess (Steve Smith) | Owned by: | |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.6.4 |
Keywords: | Cc: | cjones051073 (Chris Jones), emcrisostomo (Enrico Maria Crisostomo), mascguy (Christopher Nielsen) | |
Port: | py-tensorflow |
Description
Hitting this pip issue at destroot with Chris's most recent fixes:
sudo port install py39-tensorflow
:debug:destroot system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work/tensorflow-2.4.1" && pip-3.9 install --ignore-installed --no-cache-dir --no-dependencies --root /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work/destroot /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work/tensorflow-*.whl :info:destroot ERROR: tensorflow-2.4.1-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
Digging a little deeper into the pip creation shows this issue:
:info:build INFO: Build completed successfully, 19332 total actions :debug:build Executing proc-post-org.macports.build-build-0 :debug:build system -W /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work/tensorflow-2.4.1: ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work ... :info:build Sat Mar 27 00:18:24 EDT 2021 : === Building wheel :info:build warning: no files found matching 'README' :info:build warning: no files found matching '*.pyd' under directory '*' :info:build warning: no files found matching '*.pyi' under directory '*' :info:build warning: no files found matching '*.pd' under directory '*' :info:build warning: no files found matching '*.so.[0-9]' under directory '*' :info:build warning: no files found matching '*.dll' under directory '*' :info:build warning: no files found matching '*.lib' under directory '*' :info:build warning: no files found matching '*.csv' under directory '*' :info:build warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow' :info:build warning: no files found matching '*.proto' under directory 'tensorflow/include/tensorflow' :info:build warning: no files found matching '*' under directory 'tensorflow/include/third_party' :info:build [WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against. To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 11_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
Full log attached.
Attachments (2)
Change History (31)
comment:1 Changed 4 years ago by essandess (Steve Smith)
comment:2 Changed 4 years ago by cjones051073 (Chris Jones)
Can you compress the log to attach it please ?
Changed 4 years ago by essandess (Steve Smith)
Attachment: | main.log.xz added |
---|
comment:3 follow-up: 5 Changed 4 years ago by cjones051073 (Chris Jones)
What OS are you on b.t.w. ?
comment:4 follow-up: 6 Changed 4 years ago by cjones051073 (Chris Jones)
OK, so you are on macOS10.15. Macports configures things to use the 10.15 SDK, but it appears bazel is ignoring this and picking the 11.0 SDK you must also have installed
:info:build APPLE_SDK_PLATFORM=MacOSX \ :info:build APPLE_SDK_VERSION_OVERRIDE=11.1 \
I've had problems with bazel picking the SDK badly in the past. We need to experiment with passing the correct SDK to bazel somehow...
comment:5 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
What OS are you on b.t.w. ?
macOS 10.15.7 19H524 Xcode 12.4 12D4e xcode-select -p /Applications/Xcode.app/Contents/Developer xcrun --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk xcrun --show-sdk-version xcodebuild: error: SDK "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk" cannot be located. xcrun: error: unable to lookup item 'SDKVersion' in SDK '/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk'
Per comment:ticket:62474:23, I'll try again with
xcrun --kill-cache sudo xcode-select --switch /Library/Developer/CommandLineTools xcrun --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
But in the case here, the build works, so I'm not sanguine this will make a difference.
comment:6 follow-up: 8 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
OK, so you are on macOS10.15. Macports configures things to use the 10.15 SDK, but it appears bazel is ignoring this and picking the 11.0 SDK you must also have installed
:info:build APPLE_SDK_PLATFORM=MacOSX \ :info:build APPLE_SDK_VERSION_OVERRIDE=11.1 \I've had problems with bazel picking the SDK badly in the past. We need to experiment with passing the correct SDK to bazel somehow...
Oh, maybe sudo xcode-select --switch /Library/Developer/CommandLineTools
will make a difference. I'll try and check back in after many, many hours of tensorflow building.
comment:7 follow-up: 10 Changed 4 years ago by cjones051073 (Chris Jones)
b.t.w. you really should address the xcrun error above, as its not normal. e.g.I have on macOS11
Oberon ~/Downloads > xcode-select -p /Applications/Xcode.app/Contents/Developer Oberon ~/Downloads > xcrun --show-sdk-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk Oberon ~/Downloads > xcrun --show-sdk-version 11.1
comment:8 Changed 4 years ago by cjones051073 (Chris Jones)
Replying to essandess:
Replying to cjones051073:
OK, so you are on macOS10.15. Macports configures things to use the 10.15 SDK, but it appears bazel is ignoring this and picking the 11.0 SDK you must also have installed
:info:build APPLE_SDK_PLATFORM=MacOSX \ :info:build APPLE_SDK_VERSION_OVERRIDE=11.1 \I've had problems with bazel picking the SDK badly in the past. We need to experiment with passing the correct SDK to bazel somehow...
Oh, maybe
sudo xcode-select --switch /Library/Developer/CommandLineTools
will make a difference. I'll try and check back in after many, many hours of tensorflow building.
I doubt it will help as bazel does its own picking of the SDK internally.
Note, you don't have to wait for the entire build to run to know if it will fail. Just start the build in verbose mode and check to see what SDK is being used in messages like
:info:build exec env - \ :info:build APPLE_SDK_PLATFORM=MacOSX \ :info:build APPLE_SDK_VERSION_OVERRIDE=11.1 \ :info:build PATH=/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin \ :info:build XCODE_VERSION_OVERRIDE=12.4.0.12D4e \ :info:build external/local_config_cc/wrapped_clang ....
If it still says SDK 11.1 then you know its still a problem.
comment:9 Changed 4 years ago by cjones051073 (Chris Jones)
Note, we used to have a check for the SDK in the old port file, but it was removed as it was also causing problems in some places
comment:10 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
b.t.w. you really should address the xcrun error above, as its not normal.
Apple bug report FB9051310.
It's a known issue, unresolved to the best of my knowledge: https://stackoverflow.com/questions/64837345/force-sdk-for-make-and-xcrun-version-error
comment:11 follow-up: 12 Changed 4 years ago by cjones051073 (Chris Jones)
Just wondering, but do you get warnings like
Warning: cltversion: For a possible fix, please see: https://trac.macports.org/wiki/ProblemHotlist#reinstall-clt
?
comment:12 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
Just wondering, but do you get warnings like
Warning: cltversion: For a possible fix, please see: https://trac.macports.org/wiki/ProblemHotlist#reinstall-clt?
In the past, always correct it with:
sudo touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress && softwareupdate -l && softwareupdate -i -r && sudo rm /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
comment:13 Changed 4 years ago by cjones051073 (Chris Jones)
Another question - Do you have the shell env var SDKROOT set ? If so can you please remove this and try again.
I was able to reproduce your problem in my 10.15 VM, and in the end I traced it to having
MacVM1015 ~/Projects/MacPorts/ports > echo $SDKROOT /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
removing that I now get
MacVM1015 ~/Projects/MacPorts/ports > xcode-select -p /Applications/Xcode.app/Contents/Developer MacVM1015 ~/Projects/MacPorts/ports > xcrun --show-sdk-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk MacVM1015 ~/Projects/MacPorts/ports > xcrun --show-sdk-version 11.1
comment:14 Changed 4 years ago by cjones051073 (Chris Jones)
Of course, SDK 11.1 is still not what we want, but at least the above now works.
comment:15 Changed 4 years ago by essandess (Steve Smith)
Progress: the wheel builds, bit I hit this permissions issue on the clean command. Log attached.
:info:build INFO: Starting clean. :debug:build Attempting ln -sf /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/python/py-tensorflow/work :debug:build symlink: /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/python/py-tensorflow/work -> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work: permission denied
Build settings:
printenv SDKROOT <not set> xcode-select -p /Library/Developer/CommandLineTools xcrun --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk xcrun --show-sdk-version 10.15.6
Changed 4 years ago by essandess (Steve Smith)
Attachment: | main.log.2.xz added |
---|
comment:16 Changed 4 years ago by cjones051073 (Chris Jones)
Your build is configured differently. You are not using Xcode any more, but macports toolkit.
:info:build CC=/opt/local/bin/clang-mp-9.0 \ :info:build CXX=/opt/local/bin/clang++-mp-9.0 \ :info:build PATH=/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin \ :info:build PYTHON_BIN_PATH=/opt/local/bin/python3.9 \ :info:build PYTHON_LIB_PATH=/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages \ :info:build TF2_BEHAVIOR=1 \ :info:build TF_CONFIGURE_IOS=0 \
thats not the default on 10.15. Did you intentionally make that change ?
comment:17 follow-up: 18 Changed 4 years ago by cjones051073 (Chris Jones)
Actually I see why you are using MP's clang 9.0. I guess you explicitly set the SDK to the CLT ?
sudo xcode-select -s /Library/Developer/CommandLineTools
If I do that, then I can see the xcodeversion in the proc to check if the MP's toolkit should be used returns none, and thus MP's clang 9.0 is used.
This in itself isn't a problem, just not completely expected. I might see if I can tune that to also check the CLT version.
comment:18 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
Actually I see why you are using MP's clang 9.0. I guess you explicitly set the SDK to the CLT ?
sudo xcode-select -s /Library/Developer/CommandLineTools
Yes, because this fixes the bazel + xcrun --show-sdk-version
issues discussed above. Before setting this, the pip whl was built for the wrong sdk.
comment:19 follow-up: 20 Changed 4 years ago by cjones051073 (Chris Jones)
Right, thats fine. I'll fix the build using MP's toolkit in this case.
I have no idea about the permissions error b.t.w.
comment:20 Changed 4 years ago by essandess (Steve Smith)
Replying to cjones051073:
Right, thats fine. I'll fix the build using MP's toolkit in this case.
I have no idea about the permissions error b.t.w.
Thank you. I recall throwing in the towel on other bazel ports and setting build_as root
.
comment:21 Changed 4 years ago by cjones051073 (Chris Jones)
I've never needed that so I am not really include to add it to the port, as I suspect its some nuance of your setup, not a general issue.
comment:22 Changed 4 years ago by essandess (Steve Smith)
Re: the permission issue
For some reason, the bazel clean command is trying to link into /opt/local/var/macports/sources/…
This is causing the permissions issue. (And obviously a bad idea to build_as root
here.)
Attempting ln -sf \ /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow/py39-tensorflow/work \ /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/python/py-tensorflow/work
I'm trying to figure out if there's anything in bazel --help clean
that will stop this, or shows what causes it.
comment:23 Changed 4 years ago by essandess (Steve Smith)
Related: Please note the bazel build and clean option --macos_sdk_version
in bazel --help build
and bazel --help clean
. This might let us sidestep whatever bazel is doing to get this, and get it wrong.
comment:24 Changed 4 years ago by essandess (Steve Smith)
Also, why are we running that bazel clean step by default as part of a package manager? Everything in ${workpath
} gets cleaned anyway after activation. Would it be a better KISS design to leave as an option with of default of not doing it?
comment:25 Changed 4 years ago by cjones051073 (Chris Jones)
I have considered just removing the clean step. I originally added it as I was not sure where bazel was writing its build artifacts, so wanted to help clean up. If indeed everything is wiped with the regular port clean, then yes lets remove this.
comment:26 Changed 4 years ago by cjones051073 (Chris Jones)
comment:27 Changed 4 years ago by essandess (Steve Smith)
Thanks you Chris. TF on MacPorts now works and builds for me—first time since last summer, so this is much appreciated.
comment:28 Changed 17 months ago by mascguy (Christopher Nielsen)
Cc: | mascguy added |
---|
comment:29 Changed 17 months ago by mascguy (Christopher Nielsen)
Resolution: | → fixed |
---|---|
Status: | new → closed |
This has been fixed for some time. Closing as fixed.
Sorry, log too large.