Opened 4 years ago
Last modified 22 months ago
#61276 assigned defect
meson @0.55.3: /usr/bin/gnutar: meson-0.55.3/COPYING: implausibly old time stamp 1970-01-01 01:00:00
Reported by: | ballapete (Peter "Pete" Dyballa) | Owned by: | git@… |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.6.3 |
Keywords: | tiger leopard powerpc legacy-os | Cc: | bryancn, jmroot (Joshua Root) |
Port: | meson |
Description
The same error is reported for all files in the tape archive. Reason is that /usr/bin/gnutar
is used to outpack the archive. In the end files with this implausible time stamp are installed.
The correct programme to outpack is a recent GNU tar or /opt/local/bin/gnutar
.
Attachments (2)
Change History (36)
Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
comment:1 follow-up: 2 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)
Cc: | git@… removed |
---|---|
Owner: | set to git@… |
Status: | new → assigned |
Summary: | /usr/bin/gnutar: meson-0.55.3/COPYING: implausibly old time stamp 1970-01-01 01:00:00 → meson @0.55.3: /usr/bin/gnutar: meson-0.55.3/COPYING: implausibly old time stamp 1970-01-01 01:00:00 |
Hmm. The files in the archive have normal current timestamps when unpacked on a more modern macOS 10.13.6. I guess there is a bug in Tiger's gnutar.
comment:2 follow-up: 8 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Replying to ryandesign:
Right now it seems to be a (PPC) Tiger problem only. I don't remember whether I had the same problem in Leopard. This check might take some time…
comment:3 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Just checking the time stamps of the installed Meson
files on the Leopard volume:
root 552 /\ l -tr /Volumes/Leopard/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/mesonbuild/ total 1928 -rw-r--r-- 1 root wheel 38349 1 Jan 1970 rewriter.py -rw-r--r-- 1 root wheel 10240 1 Jan 1970 optinterpreter.py -rw-r--r-- 1 root wheel 4580 1 Jan 1970 munstable_coredata.py -rw-r--r-- 1 root wheel 51979 1 Jan 1970 mtest.py -rw-r--r-- 1 root wheel 11250 1 Jan 1970 msubprojects.py -rw-r--r-- 1 root wheel 12643 1 Jan 1970 msetup.py -rw-r--r-- 1 root wheel 32219 1 Jan 1970 mparser.py -rw-r--r-- 1 root wheel 12290 1 Jan 1970 mlog.py -rw-r--r-- 1 root wheel 23827 1 Jan 1970 mintro.py -rw-r--r-- 1 root wheel 24149 1 Jan 1970 minstall.py -rw-r--r-- 1 root wheel 7403 1 Jan 1970 minit.py -rw-r--r-- 1 root wheel 10255 1 Jan 1970 mesonmain.py -rw-r--r-- 1 root wheel 61041 1 Jan 1970 mesonlib.py -rw-r--r-- 1 root wheel 10501 1 Jan 1970 mesondata.py -rw-r--r-- 1 root wheel 11351 1 Jan 1970 mdist.py -rw-r--r-- 1 root wheel 12095 1 Jan 1970 mconf.py -rw-r--r-- 1 root wheel 11096 1 Jan 1970 mcompile.py -rw-r--r-- 1 root wheel 42990 1 Jan 1970 linkers.py -rw-r--r-- 1 root wheel 52460 1 Jan 1970 interpreterbase.py -rw-r--r-- 1 root wheel 228695 1 Jan 1970 interpreter.py -rw-r--r-- 1 root wheel 14880 1 Jan 1970 envconfig.py -rw-r--r-- 1 root wheel 2633 1 Jan 1970 depfile.py -rw-r--r-- 1 root wheel 109734 1 Jan 1970 build.py -rw-r--r-- 1 root wheel 13259 1 Jan 1970 arglist.py -rw-r--r-- 1 root wheel 0 1 Jan 1970 __init__.py -rw-r--r-- 1 root wheel 83600 17 Sep 20:32 environment.py
Looks to have the same problem…
comment:4 Changed 4 years ago by mf2k (Frank Schima)
Keywords: | powerpc legacy-os added; ppc removed |
---|
comment:5 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
There are more such packages: py38-click, py38-jinja2, py38-joblib, py38-regex, py38-tornado
.
comment:6 Changed 4 years ago by kencu (Ken)
Here are mine -- look fine -- Tiger PPC -- but I do have the gnutar
port installed, if that changes things:
tigerg5:/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/mesonbuild cunningh$ ls -la total 1928 drwxr-xr-x 39 root wheel 1326 Oct 20 19:47 . drwxr-xr-x 46 root wheel 1564 Oct 20 19:47 .. -rw-r--r-- 1 root wheel 0 Aug 15 09:27 __init__.py drwxr-xr-x 29 root wheel 986 Oct 20 19:47 __pycache__ -rw-r--r-- 1 root wheel 13259 Sep 10 09:39 arglist.py drwxr-xr-x 9 root wheel 306 Oct 20 19:47 ast drwxr-xr-x 11 root wheel 374 Oct 20 19:47 backend -rw-r--r-- 1 root wheel 109740 Oct 20 19:47 build.py drwxr-xr-x 11 root wheel 374 Oct 20 19:47 cmake drwxr-xr-x 19 root wheel 646 Oct 20 19:47 compilers -rw-r--r-- 1 root wheel 52437 Oct 20 19:47 coredata.py drwxr-xr-x 15 root wheel 510 Oct 20 19:47 dependencies -rw-r--r-- 1 root wheel 2633 Aug 15 09:27 depfile.py -rw-r--r-- 1 root wheel 14880 Sep 10 09:39 envconfig.py -rw-r--r-- 1 root wheel 83599 Oct 20 19:47 environment.py -rw-r--r-- 1 root wheel 228695 Sep 10 09:39 interpreter.py -rw-r--r-- 1 root wheel 52460 Sep 10 09:39 interpreterbase.py -rw-r--r-- 1 root wheel 43005 Oct 20 19:47 linkers.py -rw-r--r-- 1 root wheel 11096 Sep 10 09:39 mcompile.py -rw-r--r-- 1 root wheel 12095 Sep 10 09:39 mconf.py -rw-r--r-- 1 root wheel 11351 Sep 10 09:39 mdist.py -rw-r--r-- 1 root wheel 10501 Aug 15 09:27 mesondata.py -rw-r--r-- 1 root wheel 61041 Sep 10 09:39 mesonlib.py -rw-r--r-- 1 root wheel 10255 Sep 10 09:39 mesonmain.py -rw-r--r-- 1 root wheel 7403 Sep 10 09:39 minit.py -rw-r--r-- 1 root wheel 24149 Sep 10 09:39 minstall.py -rw-r--r-- 1 root wheel 23827 Sep 10 09:39 mintro.py -rw-r--r-- 1 root wheel 12290 Sep 10 09:39 mlog.py drwxr-xr-x 24 root wheel 816 Oct 20 19:47 modules -rw-r--r-- 1 root wheel 32219 Sep 10 09:39 mparser.py -rw-r--r-- 1 root wheel 12643 Sep 10 09:39 msetup.py -rw-r--r-- 1 root wheel 11250 Sep 10 09:39 msubprojects.py -rw-r--r-- 1 root wheel 51979 Sep 10 09:39 mtest.py -rw-r--r-- 1 root wheel 4580 Aug 15 09:27 munstable_coredata.py -rw-r--r-- 1 root wheel 10240 Sep 10 09:39 optinterpreter.py -rw-r--r-- 1 root wheel 38349 Aug 15 09:27 rewriter.py drwxr-xr-x 25 root wheel 850 Oct 20 19:47 scripts drwxr-xr-x 17 root wheel 578 Oct 20 19:47 templates drwxr-xr-x 6 root wheel 204 Oct 20 19:47 wrap
comment:7 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
root 322 /\ /opt/local/bin/xz -dc /opt/local/var/macports/distfiles/at-spi2-core/at-spi2-core-2.38.0.tar.xz | /opt/local/bin/gnutar --no-same-owner -vtf - | head drwxr-xr-x mgorse/users 0 2020-09-12 21:22 at-spi2-core-2.38.0/ -rw-r--r-- mgorse/users 420 2020-09-12 21:22 at-spi2-core-2.38.0/AUTHORS -rw-r--r-- mgorse/users 26530 2020-09-12 21:22 at-spi2-core-2.38.0/COPYING -rw-r--r-- mgorse/users 816 2020-09-12 21:22 at-spi2-core-2.38.0/INSTALL -rw-r--r-- mgorse/users 29 2020-09-12 21:22 at-spi2-core-2.38.0/MAINTAINERS -rw-r--r-- mgorse/users 28371 2020-09-12 21:22 at-spi2-core-2.38.0/NEWS -rw-r--r-- mgorse/users 3465 2020-09-12 21:22 at-spi2-core-2.38.0/README -rw-r--r-- mgorse/users 1741 2020-09-12 21:22 at-spi2-core-2.38.0/at-spi2-core.doap drwxr-xr-x mgorse/users 0 2020-09-12 21:22 at-spi2-core-2.38.0/atspi/ -rw-r--r-- mgorse/users 1419 2020-09-12 21:22 at-spi2-core-2.38.0/atspi/atspi-accessible-private.h C-c C-croot 323 /\ /opt/local/bin/xz -dc /opt/local/var/macports/distfiles/at-spi2-core/at-spi2-core-2.38.0.tar.xz | /usr/bin/gnutar --no-same-owner -vtf - | head drwxr-xr-x mgorse/users 0 1970-01-01 01:00:00 at-spi2-core-2.38.0/ -rw-r--r-- mgorse/users 420 1970-01-01 01:00:00 at-spi2-core-2.38.0/AUTHORS -rw-r--r-- mgorse/users 26530 1970-01-01 01:00:00 at-spi2-core-2.38.0/COPYING -rw-r--r-- mgorse/users 816 1970-01-01 01:00:00 at-spi2-core-2.38.0/INSTALL -rw-r--r-- mgorse/users 29 1970-01-01 01:00:00 at-spi2-core-2.38.0/MAINTAINERS -rw-r--r-- mgorse/users 28371 1970-01-01 01:00:00 at-spi2-core-2.38.0/NEWS -rw-r--r-- mgorse/users 3465 1970-01-01 01:00:00 at-spi2-core-2.38.0/README -rw-r--r-- mgorse/users 1741 1970-01-01 01:00:00 at-spi2-core-2.38.0/at-spi2-core.doap drwxr-xr-x mgorse/users 0 1970-01-01 01:00:00 at-spi2-core-2.38.0/atspi/ -rw-r--r-- mgorse/users 1419 1970-01-01 01:00:00 at-spi2-core-2.38.0/atspi/atspi-accessible-private.h
Same behaviour in my own account – doesnotworkforme?
comment:8 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Replying to ryandesign:
Right now it seems to be a (PPC) Tiger problem only. I don't remember whether I had the same problem in Leopard. This check might take some time…
No, it's also on PPC Leopard, Mac OS X 10.5.8
:
DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-pip/py38-pip/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-pip/pip-20.2.4.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: pip-20.2.4/AUTHORS.txt: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: pip-20.2.4/LICENSE.txt: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: pip-20.2.4/MANIFEST.in: implausibly old time stamp 1970-01-01 01:00:00 ...
comment:9 follow-up: 10 Changed 4 years ago by kencu (Ken)
So -- I guess we're making gnutar a dependency for -- what? -- using xz on older systems?
comment:10 follow-up: 11 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Replying to kencu:
So -- I guess we're making gnutar a dependency for -- what? -- using xz on older systems?
Actually gnutar
does not need to have any dependency. If it finds any of xz | bzip2 | gzip | lzip | lzma | lzop | compress
it can handle compressed tape archives. xz
should become a part of MacPorts
– how does it work on Catalina or such with xz
compressed archives? Apple does not put xz
in macOS…
A clever option would be if port
would be able to determine which of the compressors exists on the system and then decide to fetch the most efficient file – which would mean to take into account data rate of the internet connection, speed of the file system and RAM, and power of the CPU.
comment:11 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)
Cc: | bryancn added |
---|---|
Keywords: | leopard added |
Has duplicate #61819.
Replying to ballapete:
Replying to kencu:
So -- I guess we're making gnutar a dependency for -- what? -- using xz on older systems?
Actually
gnutar
does not need to have any dependency. If it finds any ofxz | bzip2 | gzip | lzip | lzma | lzop | compress
it can handle compressed tape archives.
Ken was suggesting that we should modify MacPorts base so that, when the distfile is an xz-compressed tarball, we use MacPorts gnutar instead of Tiger or Leopard's older version. However, duplicate #61819 shows that the problem is not specific to xz compression; in that ticket, it was a gz-compressed tarball. So the problem is presumably that some modern tar archives, regardless of what type of compression is later applied to them, cannot be extracted by the version of gnutar on Tiger or Leopard. Whether this is a bug in some modern tar archive creator, or whether these tar archives are valid and there is a bug in the decompression routines in Tiger and Leopard's gnutar, is unknown. It may be worthwhile for someone to investigate. If it turns out that these archives are invalid, then that bug could be fixed in whatever tool was used to create the archives, and then over time fewer invalid archives would be created as people upgrade their tools.
MacPorts handles decompression separately from untarring, so it is unlikely that the compression type has any bearing on our ability to process the uncompressed tarball inside.
Probably the only solution we have for this at the moment is that individual ports that use distfiles that have this quality should have code added to them to use port gnutar on those older systems. I don't know of a way that MacPorts base could detect this problem and do so automatically, since the port's dependencies must be determinable before the distfile has been downloaded.
xz
should become a part ofMacPorts
The request to do so is #52000 but there is no consensus that we should do so.
how does it work on Catalina or such with
xz
compressed archives? Apple does not putxz
in macOS…
When a portfile specifies use_xz yes
, MacPorts base automatically adds an extract dependency on port:xz
.
OS X 10.9 and later can decompress xz files, even though they do not include the xz binary. The request to use this facility in MacPorts base and thus avoid the dependency on port:xz
is #56237 but nothing has been done about it yet.
A clever option would be if
port
would be able to determine which of the compressors exists on the system and then decide to fetch the most efficient file – which would mean to take into account data rate of the internet connection, speed of the file system and RAM, and power of the CPU.
I see no reason to do anything of this kind. This would involve many complicated changes to MacPorts base, and I don't see what the benefit would be.
comment:12 follow-up: 13 Changed 4 years ago by kencu (Ken)
At least part of issue seems to be due to a change in python <https://docs.python.org/3/library/zipfile.html> where they have a new timestamps parameter:
New in version 3.8: The strict_timestamps keyword-only argument
And that now makes Tiger's older utils flag and error, it would seem.
Editing zipfile.py in python38 on Tiger, and changing the default to strict_timestamps=False instead of True and the ports I tried to build on Tiger work again, as you might expect.
I take it this is more or less the behaviour python had for years prior to them implementing the new check.
I am not necessarily suggesting this for MacPorts, but that is what I am doing, and my ports are all building again, so YMMV.
comment:13 follow-ups: 15 17 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Replying to kencu:
Ken,
I don't think that this is the problem. Look here:
pete 232 /\ /opt/local/bin/gnutar vtf distfiles/meson/meson-0.57.1.tar.gz drwxr-xr-x jpakkane/jpakkane 0 2021-02-20 14:21 meson-0.57.1/ -rw-r--r-- jpakkane/jpakkane 11358 2020-08-15 18:27 meson-0.57.1/COPYING -rw-r--r-- jpakkane/jpakkane 360 2021-02-09 00:20 meson-0.57.1/MANIFEST.in -rw-r--r-- jpakkane/jpakkane 1355 2021-02-20 14:21 meson-0.57.1/PKG-INFO -rw-r--r-- jpakkane/jpakkane 3246 2021-02-01 21:35 meson-0.57.1/README.md pete 233 /\ /opt/local/bin/gnutar vtf distfiles/meson/meson-0.54.1.tar.gz drwxr-xr-x jpakkane/jpakkane 0 2020-04-26 11:09 meson-0.54.1/ -rw-r--r-- jpakkane/jpakkane 11358 2016-01-23 19:52 meson-0.54.1/COPYING -rw-r--r-- jpakkane/jpakkane 415 2020-04-26 11:07 meson-0.54.1/MANIFEST.in -rw-r--r-- jpakkane/jpakkane 1357 2020-04-26 11:09 meson-0.54.1/PKG-INFO -rw-r--r-- jpakkane/jpakkane 3354 2020-04-26 11:07 meson-0.54.1/README.md
vs.
pete 237 /\ /usr/bin/gnutar zvtf distfiles/meson/meson-0.57.1.tar.gz /usr/bin/gnutar: Record size = 8 blocks drwxr-xr-x jpakkane/jpakkane 0 1970-01-01 01:00:00 meson-0.57.1/ -rw-r--r-- jpakkane/jpakkane 11358 1970-01-01 01:00:00 meson-0.57.1/COPYING -rw-r--r-- jpakkane/jpakkane 360 1970-01-01 01:00:00 meson-0.57.1/MANIFEST.in -rw-r--r-- jpakkane/jpakkane 1355 1970-01-01 01:00:00 meson-0.57.1/PKG-INFO -rw-r--r-- jpakkane/jpakkane 3246 1970-01-01 01:00:00 meson-0.57.1/README.md pete 238 /\ /usr/bin/gnutar zvtf distfiles/meson/meson-0.54.1.tar.gz /usr/bin/gnutar: Record size = 8 blocks drwxr-xr-x jpakkane/jpakkane 0 1970-01-01 01:00:00 meson-0.54.1/ -rw-r--r-- jpakkane/jpakkane 11358 1970-01-01 01:00:00 meson-0.54.1/COPYING -rw-r--r-- jpakkane/jpakkane 415 1970-01-01 01:00:00 meson-0.54.1/MANIFEST.in -rw-r--r-- jpakkane/jpakkane 1357 1970-01-01 01:00:00 meson-0.54.1/PKG-INFO -rw-r--r-- jpakkane/jpakkane 3354 1970-01-01 01:00:00 meson-0.54.1/README.md
To me this looks like Python
is not involved here. Indeed I never had a problem to upgrade py38-*
ports.
The problem is that port
is using (now) faulty tools and I have on my disk files from a time before any Mac OS existed.
pete 225 /\ /opt/local/bin/gnutar --version tar (GNU tar) 1.34 Copyright © 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Geschrieben von John Gilmore und Jay Fenlason. pete 226 /\ /usr/bin/gnutar --version tar (GNU tar) 1.14 +CVE-2006-0300 +CVE-2006-6097 Copyright (C) 2004 Free Software Foundation, Inc. This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute it under the terms of the GNU General Public License; see the file named COPYING for details. Written by John Gilmore and Jay Fenlason. Modified to support extended attributes.
comment:14 follow-up: 16 Changed 4 years ago by ballapete (Peter "Pete" Dyballa)
Port
performed a few minutes ago:
DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/nss/nss-3.62.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: nss-3.62/nss/.hg_archival.txt: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.arcconfig: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.clang-format: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.gitignore: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.hgignore: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.sancov-blacklist: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/.taskcluster.yml: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/COPYING: implausibly old time stamp 1970-01-01 01:00:00 /usr/bin/gnutar: nss-3.62/nss/Makefile: implausibly old time stamp 1970-01-01 01:00:00
while modern software works (also the complicated way):
pete 230 /\ /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/nss/nss-3.62.tar.gz' | /opt/local/bin/gnutar --no-same-owner -vtf - | head -rw-r--r-- 0/0 136 2021-02-19 11:03 nss-3.62/nss/.hg_archival.txt -rw-r--r-- 0/0 132 2021-02-19 11:03 nss-3.62/nss/.arcconfig -rw-r--r-- 0/0 1995 2021-02-19 11:03 nss-3.62/nss/.clang-format -rw-r--r-- 0/0 167 2021-02-19 11:03 nss-3.62/nss/.gitignore -rw-r--r-- 0/0 177 2021-02-19 11:03 nss-3.62/nss/.hgignore -rw-r--r-- 0/0 52 2021-02-19 11:03 nss-3.62/nss/.sancov-blacklist -rw-r--r-- 0/0 2544 2021-02-19 11:03 nss-3.62/nss/.taskcluster.yml -rw-r--r-- 0/0 18100 2021-02-19 11:03 nss-3.62/nss/COPYING -rw-r--r-- 0/0 5041 2021-02-19 11:03 nss-3.62/nss/Makefile
Gnutar
can handle all compressed tape archives without auxiliary tools and its recent version should become the tool working at least on Tiger
.
comment:15 follow-up: 24 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to ballapete:
The problem is that
port
is using (now) faulty tools and I have on my disk files from a time before any Mac OS existed.
Peter, as I said in comment:11 we have not determined which software is faulty. It could be that Tiger's /usr/bin/gnutar is faulty because it cannot deal with some valid tar archives that are now becoming popular. But it could also be that whatever software is creating these new tarballs is faulty and that the tarballs are actually invalid. If you are interested in this problem being fixed, please try to identify which of these theories is correct. If some modern utility is creating invalid tarballs, you can file a bug report with the author of that utility and explain to them in what way the archives they're creating are invalid, so that they can fix it, so that you will then be able to decompress future archives on Tiger again. Even if the new archives are technically valid, perhaps the author of the utility would be willing to revert whatever change they made that is causing this problem for Tiger's gnutar.
Whatever the cause, as I said earlier, we probably need to modify the affected ports to avoid the use of /usr/bin/gnutar on Tiger until the affected ports are updated to new versions that have tarballs that don't exhibit this problem.
comment:16 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to ballapete:
Gnutar
can handle all compressed tape archives without auxiliary tools and its recent version should become the tool working at least onTiger
.
If you mean that we should simply modify MacPorts base so that it uses /opt/local/bin/gnutar to decompress all tar files, we obviously cannot do that because then you could never install gnutar or its dependencies in the first place since they are distributed as (compressed) tar files.
comment:17 Changed 4 years ago by kencu (Ken)
As per #61819, base hard-codes /usr/bin/gnutar
.
If it used gnutar
instead of /usr/bin/gnutar
, at least on systems prior to darwin10 or something, then gnutar could be transparently be overridden by installing gnutar, which gets you past the bootstrap issue.
there are a few other workarounds in that ticket.
comment:18 Changed 3 years ago by kencu (Ken)
Cc: | jmroot added |
---|
comment:19 Changed 3 years ago by kencu (Ken)
josh, the ask here is that base not use the full path to gnutar on Tiger, but just use the bare gnutar command, so a newer version installed by the gnutar port would be found and used.
What do you think?
comment:20 follow-up: 21 Changed 3 years ago by kencu (Ken)
There is also this option for tar --warning=no-timestamp
but the tar on Tiger doesn't accept that, it appears:
$ uname -a Darwin tigerg5.local 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power Macintosh powerpc $ /usr/bin/tar --warning=no-timestamp /usr/bin/tar: unrecognized option `--warning=no-timestamp'
comment:21 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to kencu:
josh, the ask here is that base not use the full path to gnutar on Tiger, but just use the bare gnutar command, so a newer version installed by the gnutar port would be found and used.
What do you think?
Base uses whatever path to tar was specified using the configure argument --with-gnutar
, or if it wasn't specified, it finds it. On newer systems, for example, there is no /usr/bin/gnutar, so it finds /usr/bin/tar instead. If a user wishes to, a user could reconfigure MacPorts to use a gnutar installed by a separate bootstrap MacPorts installation, similar to the way you suggest they do so to use a newer libcurl.
Replying to kencu:
There is also this option for tar
--warning=no-timestamp
but the tar on Tiger doesn't accept that, it appears:
We wouldn't want to use it anyway. We don't want to suppress warnings about invalid timestamps; instead, we want tar to see that the archive contains valid timestamps.
I still suggest that what I wrote in comment:15 is the path forward for this issue.
comment:22 Changed 3 years ago by kencu (Ken)
You're probably right, but using a newer tar if it exists is a trivially simple fix that works.
All other fixes seem unlikely to happen to me, but you never know what kind of effort people might be motivated to come up with.
comment:23 Changed 3 years ago by jmroot (Joshua Root)
So I gather this is due to the use of high resolution timestamps, which gnu tar didn't support properly until version 1.15.90. The problem is not so much running the right tar if it is installed, but ensuring that it is installed without creating circular dependencies.
comment:24 follow-up: 27 Changed 3 years ago by jmroot (Joshua Root)
Replying to ryandesign:
Peter, as I said in comment:11 we have not determined which software is faulty. It could be that Tiger's /usr/bin/gnutar is faulty because it cannot deal with some valid tar archives that are now becoming popular. But it could also be that whatever software is creating these new tarballs is faulty and that the tarballs are actually invalid. If you are interested in this problem being fixed, please try to identify which of these theories is correct. If some modern utility is creating invalid tarballs, you can file a bug report with the author of that utility and explain to them in what way the archives they're creating are invalid, so that they can fix it, so that you will then be able to decompress future archives on Tiger again. Even if the new archives are technically valid, perhaps the author of the utility would be willing to revert whatever change they made that is causing this problem for Tiger's gnutar.
One source of such files seems to be the tarfile module in python's stdlib. It sets the mtime in the classic tar header to 0 when using the pax format, apparently reasoning that the pax header will take priority, which is true for newer unarchivers that support it correctly. https://github.com/python/cpython/blob/main/Lib/tarfile.py#L893
Even if a PR to set mtime in both places were accepted, the files created with older versions of the module aren't going to go away any time soon.
Changed 3 years ago by jmroot (Joshua Root)
Attachment: | tar_check.diff added |
---|
base patch to check for old gnutar and use -m
comment:25 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
Do have some better check then this?
pete 312 /\ tar zvtf /opt/local/var/macports/distfiles/py-tomli/tomli-1.2.2.tar.gz tar: Record size = 8 blocks -rw-r--r-- / 1072 1970-01-01 01:00:00 tomli-1.2.2/LICENSE -rw-r--r-- / 7994 1970-01-01 01:00:00 tomli-1.2.2/README.md -rw-r--r-- / 5106 1970-01-01 01:00:00 tomli-1.2.2/pyproject.toml -rw-r--r-- / 299 1970-01-01 01:00:00 tomli-1.2.2/tomli/__init__.py -rw-r--r-- / 21659 1970-01-01 01:00:00 tomli-1.2.2/tomli/_parser.py -rw-r--r-- / 2817 1970-01-01 01:00:00 tomli-1.2.2/tomli/_re.py -rw-r--r-- / 126 1970-01-01 01:00:00 tomli-1.2.2/tomli/_types.py -rw-r--r-- / 26 1970-01-01 01:00:00 tomli-1.2.2/tomli/py.typed -rw-r--r-- / 9089 1970-01-01 01:00:00 tomli-1.2.2/PKG-INFO pete 313 /\ gtar zvtf /opt/local/var/macports/distfiles/py-tomli/tomli-1.2.2.tar.gz -rw-r--r-- 0/0 1072 2021-10-25 11:50 tomli-1.2.2/LICENSE -rw-r--r-- 0/0 7994 2021-10-25 11:50 tomli-1.2.2/README.md -rw-r--r-- 0/0 5106 2021-10-25 11:50 tomli-1.2.2/pyproject.toml -rw-r--r-- 0/0 299 2021-10-25 11:50 tomli-1.2.2/tomli/__init__.py -rw-r--r-- 0/0 21659 2021-10-25 11:50 tomli-1.2.2/tomli/_parser.py -rw-r--r-- 0/0 2817 2021-10-25 11:50 tomli-1.2.2/tomli/_re.py -rw-r--r-- 0/0 126 2021-10-25 11:50 tomli-1.2.2/tomli/_types.py -rw-r--r-- 0/0 26 2021-10-25 11:50 tomli-1.2.2/tomli/py.typed -rw-r--r-- 0/0 9089 1970-01-01 01:00 tomli-1.2.2/PKG-INFO pete 314 /\ gzip -dc /opt/local/var/macports/distfiles/py-tomli/tomli-1.2.2.tar.gz | tar vtf - -rw-r--r-- / 1072 1970-01-01 01:00:00 tomli-1.2.2/LICENSE -rw-r--r-- / 7994 1970-01-01 01:00:00 tomli-1.2.2/README.md -rw-r--r-- / 5106 1970-01-01 01:00:00 tomli-1.2.2/pyproject.toml -rw-r--r-- / 299 1970-01-01 01:00:00 tomli-1.2.2/tomli/__init__.py -rw-r--r-- / 21659 1970-01-01 01:00:00 tomli-1.2.2/tomli/_parser.py -rw-r--r-- / 2817 1970-01-01 01:00:00 tomli-1.2.2/tomli/_re.py -rw-r--r-- / 126 1970-01-01 01:00:00 tomli-1.2.2/tomli/_types.py -rw-r--r-- / 26 1970-01-01 01:00:00 tomli-1.2.2/tomli/py.typed -rw-r--r-- / 9089 1970-01-01 01:00:00 tomli-1.2.2/PKG-INFO pete 315 /\ gzip -dc /opt/local/var/macports/distfiles/py-tomli/tomli-1.2.2.tar.gz | gtar vtf - -rw-r--r-- 0/0 1072 2021-10-25 11:50 tomli-1.2.2/LICENSE -rw-r--r-- 0/0 7994 2021-10-25 11:50 tomli-1.2.2/README.md -rw-r--r-- 0/0 5106 2021-10-25 11:50 tomli-1.2.2/pyproject.toml -rw-r--r-- 0/0 299 2021-10-25 11:50 tomli-1.2.2/tomli/__init__.py -rw-r--r-- 0/0 21659 2021-10-25 11:50 tomli-1.2.2/tomli/_parser.py -rw-r--r-- 0/0 2817 2021-10-25 11:50 tomli-1.2.2/tomli/_re.py -rw-r--r-- 0/0 126 2021-10-25 11:50 tomli-1.2.2/tomli/_types.py -rw-r--r-- 0/0 26 2021-10-25 11:50 tomli-1.2.2/tomli/py.typed -rw-r--r-- 0/0 9089 1970-01-01 01:00 tomli-1.2.2/PKG-INFO
comment:26 Changed 3 years ago by kencu (Ken)
It is possible to construct a much-pared-down version of gnutar, sort of "gnutar-bootstrap", that has a much lower burden of deps for installation... it might have had no deps, if I recall, or at least had very few, and nothing circular.
I built such a version of gnutar a few months ago when I was considering how to solve this problem, but I can't at the moment locate it. But that path was fruitful, should anyone care to recreate that idea.
comment:27 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
Replying to jmroot:
Replying to ryandesign:
Peter, as I said in comment:11 […]
As far as I understand this, those who create the Python module TAR file archives would need to change something that is obviously working for the vast majority. It would be easier when the few with old gear update their's. One option is to manually substitute
-rwxr-xr-x 2 root wheel 186088 24 Okt 2008 /usr/bin/gnutar -rwxr-xr-x 2 root wheel 186088 24 Okt 2008 /usr/bin/tar
with
-rwxr-xr-x 1 root admin 490120 18 Sep 11:07 /opt/local/bin/gnutar lrwxr-xr-x 1 root admin 21 18 Sep 11:07 /opt/local/bin/gtar -> /opt/local/bin/gnutar
or simply touch all files after untarring them in the build process. Or just use MacPorts' gnutar
instead when it was found that it exists. The list of Python modules that need this special treatment will be discovered…
comment:28 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
What can I do with tar_check.diff
? And when?
comment:29 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
When upgrading py-bootstrap-modules
:
DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-bootstrap-modules/py-bootstrap-modules/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-bootstrap-modules/build-0.7.0.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: build-0.7.0/LICENSE: implausibly old time stamp 1970-01-01 01:00:00 … DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-bootstrap-modules/py-bootstrap-modules/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-bootstrap-modules/flit_core-3.5.0.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: flit_core-3.5.0/build_dists.py: implausibly old time stamp 1970-01-01 01:00:00 … DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-bootstrap-modules/py-bootstrap-modules/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-bootstrap-modules/pep517-0.12.0.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: pep517-0.12.0/.bumpversion.cfg: implausibly old time stamp 1970-01-01 01:00:00 … DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-bootstrap-modules/py-bootstrap-modules/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-bootstrap-modules/tomli-1.2.2.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: tomli-1.2.2/LICENSE: implausibly old time stamp 1970-01-01 01:00:00 … DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-bootstrap-modules/py-bootstrap-modules/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-bootstrap-modules/wheel-0.37.0.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: wheel-0.37.0/LICENSE.txt: implausibly old time stamp 1970-01-01 01:00:00 …
comment:30 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-jinja2/py38-jinja2/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-jinja2/Jinja2-3.0.3.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: Jinja2-3.0.3/CHANGES.rst: implausibly old time stamp 1970-01-01 01:00:00 … DEBUG: system: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_python_py-mako/py38-mako/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/py-mako/Mako-1.1.6.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - /usr/bin/gnutar: Mako-1.1.6/AUTHORS: implausibly old time stamp 1970-01-01 01:00:00 …
comment:31 Changed 3 years ago by jmroot (Joshua Root)
We know that all tarballs generated with python's tarfile module will have this problem, so there's really no need to list every specific example.
comment:32 Changed 3 years ago by jmroot (Joshua Root)
comment:33 Changed 22 months ago by ballapete (Peter "Pete" Dyballa)
This fault is more numerous than ever, because the Python folks are updating their packages as if tomorrow will be another day.
comment:34 Changed 22 months ago by kencu (Ken)
Does https://github.com/python/cpython/pull/29693 being committed mean that at some point this problem will just disappear, when newer pythons are being used more commonly?
Main.log from PPC Tiger, Mac OS X 10.4.11