Opened 11 years ago

Closed 7 years ago

Last modified 5 years ago

#41849 closed defect (worksforme)

port 2.2.1: Error: org.macports.archivefetch for port libiconv returned: Failed to fetch signature for archive: The requested URL returned error: 400 Bad Request

Reported by: itslaksh@… Owned by: admin@…
Priority: Normal Milestone:
Component: server/hosting Version: 2.2.1
Keywords: Cc: vincent.emanuele@…, nerdling (Jeremy Lavergne), mkae (Marko Käning)
Port:

Description (last modified by ryandesign (Ryan Carsten Schmidt))

Lakshmis-MacBook-Pro:~ lreguna$ sudo port -fp install gettext
Password:
--->  Computing dependencies for gettext
--->  Dependencies to be installed: libiconv
--->  Fetching archive for libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2 from http://packages.macports.org/libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 from http://packages.macports.org/libiconv
Error: org.macports.archivefetch for port libiconv returned: Failed to fetch signature for archive: The requested URL returned error: 400 Bad Request
Error: Failed to install libiconv
Please see the log file for port libiconv for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_textproc_libiconv/libiconv/main.log
Error: The following dependencies were not installed: libiconv
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port gettext failed
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

Attachments (5)

main.log (2.4 KB) - added by itslaksh@… 11 years ago.
log file
libiconv.debug (4.0 KB) - added by vincent.emanuele@… 11 years ago.
Output of the command: $ sudo port -d install libiconv > libiconv.debug 2>&1
tcp_dump_filtered.pcap (1.4 KB) - added by vincent.emanuele@… 11 years ago.
Output of running command: $ sudo tcpdump -i en0 -w macports-tcpdump.pcap
otool_pextlib.output (366 bytes) - added by vincent.emanuele@… 11 years ago.
Output of command: $ otool -L /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib
macports-tcpdump.pcap (1.7 MB) - added by vincent.emanuele@… 11 years ago.
Output of command: $ sudo tcpdump -w macports-tcpdump.pcap host packages.macports.org

Download all attachments as: .zip

Change History (24)

Changed 11 years ago by itslaksh@…

Attachment: main.log added

log file

comment:1 Changed 11 years ago by itslaksh@…

Cc: itslaksh@… added

Cc Me!

comment:2 Changed 11 years ago by vincent.emanuele@…

Cc: vincent.emanuele@… added

Cc Me!

comment:3 Changed 11 years ago by vincent.emanuele@…

I just reviewed this ticket and I have the exact same problem. I've been researching this for 1.5 days with no success. Here is a summary of what I've noticed.

Setup:

Symptoms

  • I'm experiencing exactly the same broken libiconv issue that itslaksh@… pointed out. My main.log file looks very similar.
  • Other packages break too. Confirmed the same error message for:
    • All packages that depend on libiconv (there are a lot of them, and they fail at the libiconv install step)
    • openssl
  • However, the following commands/ports install okay
    • sudo port selfupdate
    • sudo port install zlib
    • bzip2
    • libpng
    • freetype
  • I've attached the output of running $ sudo port -d install libiconv as the attached file libiconv.debug to the ticket

Things I've tried

  • Wipe everything and reinstall
    • Uninstalled Macports according to the directions: https://guide.macports.org/chunked/installing.macports.uninstalling.html
    • Uninstalled Xcode by moving /Applications/Xcode.app to the Trash bin
    • Installed Xcode (I've tried 2 ways, from developer.apple.com & from the App store with same result)
    • Accepted Xcode license, reinstalled command-line tools
    • Reinstalled Macports from binary (link given above)
    • Problem completely reproduced
  • Look for related trac tickets in Macports (search for "bad request")
    • Tickets: 41849 (this ticket), 39949, 33715, 28590, and 15901
    • None of them seem particularly related
  • Google: googled for macports "bad request"
    • Looked through several hits, was not able to see what actions I needed to take to resolve this
  • Looked at the code
    • It looks like the problem occurs in this file: /opt/local/share/macports/Tcl/package1.0/portarchivefetch.tcl
      • Lines 242 - 245
    • I could not figure out the cause after putting some debug statements in that script

Please let me know if there's anything I can do to help you all diagnose and resolve this outstanding issue, as it's crippled my ability to use macports and I'd love to resolve this ASAP!

Last edited 11 years ago by vincent.emanuele@… (previous) (diff)

Changed 11 years ago by vincent.emanuele@…

Attachment: libiconv.debug added

Output of the command: $ sudo port -d install libiconv > libiconv.debug 2>&1

comment:4 Changed 11 years ago by nerdling (Jeremy Lavergne)

Cc: snc@… added

From the log, you can piece together the URL that MacPorts is attempting: http://packages.macports.org/libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160

When I ask curl for a HEAD request of that, I get 200 OK:

curl -I http://packages.macports.org/libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160

If you run the curl command do you also get 200, or 400?

Last edited 11 years ago by nerdling (Jeremy Lavergne) (previous) (diff)

comment:5 Changed 11 years ago by neverpanic (Clemens Lang)

To further debug this, can you run tcpdump while trying to download the file like this:

sudo /usr/sbin/tcpdump -w macports-tcpdump.pcap host packages.macports.org

then open a new shell and attempt to install libiconv. After it failed, press Ctrl+C to stop recording packages and attach the file macports-tcpdump.pcap.

Please also provide the output of otool -L /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib.

Maybe MacPorts should also try the other mirrors for the signature file rather than failing when the first mirrors fails to serve it.

Changed 11 years ago by vincent.emanuele@…

Attachment: tcp_dump_filtered.pcap added

Output of running command: $ sudo tcpdump -i en0 -w macports-tcpdump.pcap

Changed 11 years ago by vincent.emanuele@…

Attachment: otool_pextlib.output added

Output of command: $ otool -L /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib

comment:6 Changed 11 years ago by mkae (Marko Käning)

Cc: mk@… added

Cc Me!

comment:7 Changed 11 years ago by neverpanic (Clemens Lang)

Can you provide the unmodified pcap with the host packages.macports.org filter as described above? I'd like to open it in Wireshark and make sure there's no binary junk or weird packages being sent and that's not really possible with the textual output you've attached. Using the filter should ensure there's no private data in the file either.

The otool output looks fine.

Are you currently in a country that might be affected by US export restrictions, such as Iran, etc.? Are you behind a (corporate) proxy server?

Changed 11 years ago by vincent.emanuele@…

Attachment: macports-tcpdump.pcap added

Output of command: $ sudo tcpdump -w macports-tcpdump.pcap host packages.macports.org

comment:8 Changed 11 years ago by vincent.emanuele@…

Thanks for your quick responses! I just attached the unmodified pcap file.

I am in the United States, and we've had a couple new hires within the past few weeks set up macports successfully on similar hardware. Since they don't have the same problem, it feels like it must be a local setting on my computer or a config issue?

comment:9 Changed 11 years ago by neverpanic (Clemens Lang)

It's a legitimate bad request, because it's using HTTP/1.1 but doesn't send a Host header. I should have paid more attention to your filtered output, it also contains the same problem. The headers for the first GET request are:

GET /libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2 HTTP/1.1
User-Agent: MacPorts 2.2.1 libcurl/7.30.0
Host: packages.macports.org
Accept: */*

but the second GET doesn't send Accept or Host:

GET /libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 HTTP/1.1
User-Agent: MacPorts 2.2.1 libcurl/7.30.0

I wonder why it doesn't and whether we can do something to force curl to send the headers, but it's too late to investigate further now.

comment:10 Changed 11 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: itslaksh@… removed
Description: modified (diff)

comment:11 Changed 11 years ago by vincent.emanuele@…

According to http://stackoverflow.com/questions/356705/how-to-send-a-header-using-a-http-request-through-a-curl-call, it looks like we could force curl to put the Host and Accept in the header by adding:

curl --header "Host: packages.macports.org" --header "Accept: */*" <keep the rest of the curl call the same>

This may be a band-aid on the underlying problem, but I'll take it if it works.

Do you have any suggestions for work-arounds? I suppose manually downloading the package is an idea?

comment:12 Changed 11 years ago by neverpanic (Clemens Lang)

MacPorts does not use the curl executable, but links against libcurl and wraps the C interface in a few different Tcl commands. I think the problem might be related to source:trunk/base/src/pextlib1.0/curl.c@117017:369-375#L365, although the documentation says this should strip the existing header, and it would expect why it works the first time (where the exact same code is being executed).

I think this might actually be a bug in curl.

comment:13 Changed 11 years ago by neverpanic (Clemens Lang)

Manually downloading might certainly work, yes. You can try that, but it's tedious, if it keeps failing with other packages. You could also try a different mirror by removing packages.macports.org from the file at $(port dir yubico-pam)/../../_resources/port1.0/fetch/archive_sites.tcl. If that helps, we can draft the necessary directives to put into macports.conf to make this change permanent (otherwise the next selfupdate will revert it).

comment:14 Changed 11 years ago by vincent.emanuele@…

I removed packages.macports.org from the file you suggested. The install is working now.

$ sudo port install libiconv
--->  Fetching archive for libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 from http://mse.uk.packages.macports.org/sites/packages.macports.org/libiconv
--->  Installing libiconv @1.14_0
--->  Activating libiconv @1.14_0
--->  Cleaning libiconv
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.
$ sudo port install openssl
--->  Computing dependencies for openssl
--->  Fetching archive for openssl
--->  Attempting to fetch openssl-1.0.1f_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/openssl
--->  Attempting to fetch openssl-1.0.1f_0.darwin_13.x86_64.tbz2.rmd160 from http://mse.uk.packages.macports.org/sites/packages.macports.org/openssl
--->  Installing openssl @1.0.1f_0
--->  Activating openssl @1.0.1f_0
--->  Cleaning openssl
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

So yes, please! What do I need to do to make sure this change is permanent? I really depend on MacPorts for all kinds of command-line goodness at work. Thanks very much for your response.

comment:15 Changed 11 years ago by neverpanic (Clemens Lang)

Adding host_blacklist packages.macports.org to your macports.conf in /opt/local/etc/macports/ should make MacPorts ignore this server forever.

comment:16 Changed 10 years ago by jmroot (Joshua Root)

Owner: changed from wsiegrist@… to admin@…

comment:17 Changed 8 years ago by ryandesign (Ryan Carsten Schmidt)

Owner: changed from admin@… to admin@…
Status: newassigned

comment:18 Changed 7 years ago by raimue (Rainer Müller)

Resolution: worksforme
Status: assignedclosed

I doubt this is still a problem after switching the hosting site for packages.macports.org.

comment:19 Changed 5 years ago by ryandesign (Ryan Carsten Schmidt)

A similar problem was reported against a different server in #43689. We didn't get enough information to know if the cause was the same, and we still don't know what we could do about it. Both reports were on Mavericks, so maybe that indicates a bug in the Mavericks version of the curl library.

Note: See TracTickets for help on using tickets.