Opened 7 months ago

Closed 4 months ago

Last modified 2 months ago

#69555 closed enhancement (fixed)

Use cloning to activate port images on APFS

Reported by: jmroot (Joshua Root) Owned by: jmroot (Joshua Root)
Priority: Normal Milestone: MacPorts 2.10.0
Component: base Version:
Keywords: Cc:
Port:

Description

Background on why things are the way they currently are: #13143, #13601, #19458

Our current way of handling port images predates APFS by many years. Some of the problems it was intended to solve may no longer be relevant on modern systems.

Storing port images as extracted directories and using COPYFILE_CLONE to activate their files could improve performance a lot. Cloning rather than hard linking (as was done historically) would prevent changes to the activated files from affecting those in the image, because of the COW semantics. Hopefully Time Machine also handles APFS clones better than it historically handled hard links.

Since a lot of installations are done from downloaded archives anyway, it probably makes sense to always create an intermediate archive when installing from a local destroot. This handily solves the hard link detection problem and is also one of the easiest ways of applying HFS compression. We could have an option to keep the archive as well if desired (such as on the buildbot).

We would of course have to fall back to the current method on non-APFS filesystems, and when the image directory is on a different volume than the activated location.

Change History (7)

comment:1 in reply to:  description ; Changed 7 months ago by ryandesign (Ryan Carsten Schmidt)

Replying to jmroot:

when the image directory is on a different volume than the activated location.

The activated location could span multiple volumes.

comment:2 in reply to:  1 Changed 7 months ago by jmroot (Joshua Root)

Replying to ryandesign:

The activated location could span multiple volumes.

We already behave suboptimally in that case, so no regression.

comment:3 Changed 4 months ago by jmroot (Joshua Root)

Owner: set to jmroot
Resolution: fixed
Status: newclosed

In 4504942d2283f2950d273b0aee93b9ed867007ea/macports-base (master):

Use clonefile for port images, and detect hardlinks

New portimage_mode config option controls whether the extracted
directory, the archive, or both are kept for port images. Default is
directory if the filesystem that the software directory is on supports
cloning. Otherwise the default is archive.

In directory mode, downloaded archives are extracted only once.
Subsequent activations just clone the files. If activation is across
devices for some files, fall back to copying for those files.

In all cases, hard links are now recreated correctly whenever possible
even if the activated location is on a different device than the port
image.

Closes: #69555

comment:4 Changed 4 months ago by jmroot (Joshua Root)

Milestone: MacPorts Future

comment:5 Changed 3 months ago by jmroot (Joshua Root)

Milestone: MacPorts FutureMacPorts 2.10.0

comment:6 Changed 2 months ago by barracuda156

I have an impression that image_to_archives script freezes now. Has this been tested on 10.5, for example?

  1. S. It installs fine if there are no ports installed, but either takes hours or just does not work at all when many ports are installed, and MacPorts is been upgraded.
Last edited 2 months ago by barracuda156 (previous) (diff)

comment:7 in reply to:  6 Changed 2 months ago by jmroot (Joshua Root)

Replying to barracuda156:

Has this been tested on 10.5, for example?

Yes. If you want to report a bug, open a new ticket with steps to reproduce.

Note: See TracTickets for help on using tickets.