#58882 closed defect (worksforme)
Python37 configure script segfaults in generate-posix-vars on macOS 10.15
Reported by: | vdoublie | Owned by: | jmroot (Joshua Root) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | |
Keywords: | catalina | Cc: | colons (colons), trainman75 |
Port: | python37 |
Description
During the build of Python37 under macOS10.15, generate-posix-vars fails. This does not happens under 10.14
just "forcing" the MACOSX_DEPLOYMENT_TARGET variable to 10.14 during the build of python37 instead of 10.15 "fixes" the problem.
See main_macports.log for errors
See main_patched.log
Attachments (4)
Change History (51)
Changed 5 years ago by vdoublie
Attachment: | main_macport.log added |
---|
comment:1 Changed 5 years ago by mf2k (Frank Schima)
Keywords: | catalina added; upcoming macOS release build rules removed |
---|---|
Owner: | set to jmroot |
Status: | new → assigned |
comment:3 Changed 5 years ago by Schamschula (Marius Schamschula)
Are you sure you uploaded the main_patched.log. I see the same MACOSX_DEPLOYMENT_TARGET and the same error.
comment:4 Changed 5 years ago by vdoublie
oops forgot about the "autoclean" done at the end of the successful install here it is
Changed 5 years ago by vdoublie
Attachment: | main_patched.log added |
---|
forcing MACOSX_DEPLOYMENT_TARGET to 10.14 so generate-posix-vars knows what to do
comment:7 Changed 5 years ago by jmroot (Joshua Root)
Summary: | Python37 configure script requires an update of MACOSX_DEPLOYMENT_TARGET rules for upcoming macOS 10.15 → Python37 configure script segfaults in generate-posix-vars on macOS 10.15 |
---|
Upstream python developers who are disclosed on Catalina would probably like to see the crash log.
comment:8 Changed 5 years ago by josh-blake (Josh Blake)
Python will build if you use the Xcode 10.3 toolchain - either select 10.3 (10G8) from within Xcode, or point appropriately using
xcode-select -s /Applications/Xcode.app/Contents/Developer
from the command line. You would also be wise to run a
sudo port select clang none sudo port select llvm none
just to force macports to use Apples toolchain.
comment:9 Changed 5 years ago by pmetzger (Perry E. Metzger)
jmroot, do we have any progress on this? Catalina is out, and python 3.7 is a critical package...
comment:10 Changed 5 years ago by jmroot (Joshua Root)
I have no idea if it's still an issue with the final release of Catalina and current Xcode.
comment:11 Changed 5 years ago by colons (colons)
Running the App Store version of Xcode 11.1, python37 runs into this. So does python27.
:info:build DYLD_FRAMEWORK_PATH=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python37/python37/work/Python-3.7.4 ./python.exe -E -S -m sysconfig --generate-posix-vars ;\ :info:build if test $? -ne 0 ; then \ :info:build echo "generate-posix-vars failed" ; \ :info:build rm -f ./pybuilddir.txt ; \ :info:build exit 1 ; \ :info:build fi :info:build /bin/sh: line 1: 25214 Segmentation fault: 11 DYLD_FRAMEWORK_PATH=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python37/python37/work/Python-3.7.4 ./python.exe -E -S -m sysconfig --generate-posix-vars
:info:build DYLD_FRAMEWORK_PATH=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27 /work/Python-2.7.16 ./python.exe -E -S -m sysconfig --generate-posix-vars ;\ :info:build if test $? -ne 0 ; then \ :info:build echo "generate-posix-vars failed" ; \ :info:build rm -f ./pybuilddir.txt ; \ :info:build exit 1 ; \ :info:build fi :info:build /bin/sh: line 1: 31630 Segmentation fault: 11 DYLD_FRAMEWORK_PATH=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.16 ./python.exe -E -S -m sysconfig --generate-posix-vars :info:build generate-posix-vars failed
I've tried selecting Xcode 10.3, cleaning, and rebuilding, but that doesn't seem to change anything.
comment:12 follow-up: 15 Changed 5 years ago by jmroot (Joshua Root)
Did anyone ever report it upstream?
comment:13 Changed 5 years ago by colons (colons)
With Xcode 10.3 and 11.1, I'm able to build, install, and run from the 3.7.4 tarball successfully, with no modifications. The below step goes by with no segfault:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.7m.a(pymath.o) has no symbols gcc -Wl,-stack_size,1000000 -framework CoreFoundation -o python.exe Programs/python.o libpython3.7m.a -ldl -framework CoreFoundation ./python.exe -E -S -m sysconfig --generate-posix-vars ;\ if test $? -ne 0 ; then \ echo "generate-posix-vars failed" ; \ rm -f ./pybuilddir.txt ; \ exit 1 ; \ fi gcc -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -I. -I./Include -DPy_BUILD_CORE -o Modules/_math.o Modules/_math.c
I can't find anyone reporting this particular issue in Python's issue tracker. 38295 isn't the same issue and I can't see anything that looks like this in their issue tracker.
comment:14 Changed 5 years ago by colons (colons)
Cc: | colons added |
---|
comment:15 Changed 5 years ago by yan12125 (Chih-Hsuan Yen)
Cc: | yan12125 added |
---|
Replying to jmroot:
Did anyone ever report it upstream?
Here is one related comment: https://bugs.python.org/issue38014#msg351312. It says --enable-framework is related to the crash.
comment:16 Changed 5 years ago by jmroot (Joshua Root)
OK, so can someone with the final version of Catalina please show a build log with the failure, and the crash log?
comment:17 Changed 5 years ago by trainman75
Cc: | trainman75 added |
---|
comment:18 Changed 5 years ago by trainman75
Build log added. If you need anything else, let me know (and preferably how to get it in the form most useful to you!)
comment:19 Changed 5 years ago by jmroot (Joshua Root)
I still need the crash log please. Should be in either ~/Library/Logs/DiagnosticReports
or /Library/Logs/DiagnosticReports
.
comment:20 Changed 5 years ago by mf2k (Frank Schima)
I cannot reproduce this on the release versions of Catalina and Xcode 11.1.
$ port -v installed python37 The following ports are currently installed: python37 @3.7.4_1 (active) platform='darwin 19' archs='x86_64' date='2019-10-11T03:29:47-0600'
comment:21 Changed 5 years ago by jmroot (Joshua Root)
Interesting. Have those seeing the crash started fresh with 10.15 final, or could there be bits left over from the beta?
comment:22 Changed 5 years ago by vdoublie
I have been able to build python37 on Catalina Xcode 11.1 without crash.
Changed 5 years ago by trainman75
Attachment: | python.exe_2019-10-12-002028_MacBook-Pro.crash added |
---|
comment:24 Changed 5 years ago by kencu (Ken)
my guess is that this is another manifestation of
<https://github.com/macports/macports-ports/commit/6e78e5c9495b4dc4e7e050fae2b41dd5b9accfdd>
comment:27 Changed 5 years ago by jmroot (Joshua Root)
@trainman75 the only significant difference between your log and mine that I can see up to the point where yours fails is:
-:info:configure checking for python3.7... python3.7 +:info:configure checking for python3.7... no +:info:configure checking for python3... python3
How is configure finding python3.7 on your system? Also you have coreutils installed; clutching at straws but I don't suppose deactivating that port makes any difference?
comment:28 Changed 5 years ago by jmroot (Joshua Root)
Well, having coreutils installed made no difference here. So I guess we're back to the admittedly vague "beta pollution" hypothesis, unless someone can repro on a clean never-beta 10.15 installation.
comment:29 Changed 5 years ago by vdoublie
I sort of did: I kept my "old port installed under Mojave + Xcode 10 stuff" and it is still working under Catalina... as long as I don't touch it. SO I moved out of the way the "old Mojave" stuff (Macports directory, and /opt/local the rebuild macports using GitHub (so I am up to date port wise" then just port install python37 - so it has to rebuild everything from scratch and it builds no sweat... but it is only very recent. So it suspect that this is a port that does not like to "see beta or pre-release" or anything that is not "user/public release" as I submitted forcing MACOSX_DEPLOYMENT_TARGETto 10.14 (so mascaraing it) in the makefile.pre did the trick... for just that part of the code... if the target is 10.15 up until this special part of the make then reset just after ... it build also as should be... so It really a PB of the generate_posix_var that cannot find a rule somewhere for "unknown forward versions of the platform"
comment:30 Changed 5 years ago by trainman75
I just tried adding the lines mentioned in the commit from comment:24, and success. Python37 builds where it didn't before.
With nothing more than an educated guess, I placed them just before "build.target all"
I had been running Catalina beta and Xcode11 beta, but the build issue was present in that configuration. Updated both to public release in the last few days, still no luck. Reinstalled MacPorts from binary, still no go. After that stage, I had uninstalled all ports, but realise now I'd forgotten to do the various directory removals from the uninstall guide, so I may still have had a dirty setup.
Hope this helps track down a fix.
comment:31 follow-up: 34 Changed 5 years ago by kencu (Ken)
It could be processor dependent as well, so perhaps not everyone would see it. Or some other random variation to be discovered.
comment:32 Changed 5 years ago by 19h
I can verify that -fno-stack-check successfully works around the segmentation fault, for both python37 and python27. Device: MacBookPro13,3 using build 19A582a (Catalina GM).
comment:34 Changed 5 years ago by jmroot (Joshua Root)
Replying to kencu:
It could be processor dependent as well, so perhaps not everyone would see it. Or some other random variation to be discovered.
It's very easy to disprove my hypothesis if false: someone who sees the issue just needs to make a fresh Catalina installation on the same hardware and repro.
comment:35 Changed 5 years ago by jmroot (Joshua Root)
Python upstream can't repro either. https://bugs.python.org/issue38014#msg354598
They've asked that if the issue is still present, someone who can reproduce it should open a new issue on their bug tracker and show them how to do it.
comment:36 Changed 5 years ago by dlamija (Muhammed Ramiza)
Bulid sucessfully on full release Catalina & XCode 11. No crash. For those whose failed maybe there is some leftover from beta version
comment:37 Changed 5 years ago by pcallycat
Still unable to build, Catalina, Xcode 11. Clean install of MacPorts and Xcode production releases on general release of MacOS, with a new user just to cover any missed directories. Tomorrow I'll do a full clean install of MacOS. MacBook Pro late 2013 model.
edit: rather than reinstall my whole machine, I just added -fno-stack-check as mentioned prior to build. Not sure why this is necessary.
comment:38 Changed 5 years ago by yan12125 (Chih-Hsuan Yen)
Cc: | yan12125 removed |
---|
comment:39 follow-up: 40 Changed 5 years ago by mouse07410 (Mouse)
Pardon my ignorance, but (a) where did you add -fno-stack-check
, and (b) does build with Xcode-11.3 still exhibit this problem?
comment:40 Changed 5 years ago by jmroot (Joshua Root)
Replying to mouse07410:
(b) does build with Xcode-11.3 still exhibit this problem?
Anyone able to answer this? If there's no issue with 11.3 then I think we can just close the ticket.
comment:41 Changed 5 years ago by jmroot (Joshua Root)
Resolution: | → worksforme |
---|---|
Status: | assigned → closed |
comment:42 Changed 5 years ago by mouse07410 (Mouse)
I've installed (via Macports) python37
and python38
on Catalina 10.15.3 with Xcode-11.3.1. I'd say this problem has been remedied. Would appreciate others confirm though...
comment:43 Changed 4 years ago by fredowski
I have exactly the same problem but with python38 on MacOS 10.13.6 with MACOSX_DEPLOYMENT_TARGET set to 10.7. It works with target set to 10.8
:info:build DYLD_FRAMEWORK_PATH=/opt/macports/install/var/macports/build/_opt_macports_macp orts-ports_lang_python38/python38/work/Python-3.8.5 ./python.exe -E -S -m sysconfig --gener ate-posix-vars ;\ :info:build if test $? -ne 0 ; then \ :info:build echo "generate-posix-vars failed" ; \ :info:build rm -f ./pybuilddir.txt ; \ :info:build exit 1 ; \ :info:build fi :info:build /bin/sh: line 1: 32838 Killed: 9 DYLD_FRAMEWORK_PATH=/opt/macport s/install/var/macports/build/_opt_macports_macports-ports_lang_python38/python38/work/Pytho n-3.8.5 ./python.exe -E -S -m sysconfig --generate-posix-vars :info:build generate-posix-vars failed :info:build make: *** [pybuilddir.txt] Error 1
comment:44 Changed 4 years ago by fredowski
I checked some other MACOSX_DEPLOYMENT_TARGETS and this is the result on MacOS 10.13 with Xcode 10.1:
- 10.5 works
- 10.6 failure
- 10.7 failure
- 10.8 works
So you can build python38 with deployment target 10.5 but not 10.6 and not 10.7. Strange.
comment:45 Changed 4 years ago by kencu (Ken)
I wonder if this might be an xcode clang thing. Perhaps building it with one of MacPorts' clang compilers would work.
python38 does build on 10.6 and 10.7, of course, when building directly on those systems.
comment:46 Changed 4 years ago by fredowski
I tried with gcc10 but that fails also. Maybe it is related to the sdk.
comment:47 Changed 4 years ago by kencu (Ken)
always still with the same mysterious Catalina error?
Error Code: 0x00000014 (no mapping for user instruction write)
In the future, please add the port maintainer(s) to Cc (
port info --maintainers python37
), if any.