hdf5 / field3d / openimageio dylib mismatch on builders; causes blender builds to fail

[jasonliu-- (Jason Liu)]

There appears to be a dylib version mismatch between a couple of the packages in the ports tree. As a result, all of the ​builds for blender are failing, with the following error:

dyld: Library not loaded: /opt/local/lib/libhdf5.103.dylib
  Referenced from: /opt/local/lib/libOpenImageIO.2.1.dylib
  Reason: image not found

OpenImageIO is dependent on hdf5, not directly, but by way of depending on field3d.

I encountered this same error on my own machine, and did some troubleshooting. It appears to have something to do with this PR, which updated the hdf5 port to version 1.12.0:

​https://github.com/macports/macports-ports/pull/7983

To get the error to go away on my machine, I had to reactivate hdf5 @1.10.7. Running port contents on hdf5, it turns out that libhdf5.103.dylib exists in hdf5 @1.10.7, but doesn't exist in hdf5 @1.12.0. Instead, hdf5 @1.12.0 has libhdf5.200.dylib.

I'm not really sure how to fix this on the builders though, since PR7983 included a revbump of field3d to revision 11, which should have caused it to get rebuilt against hdf5 @1.12.0, if I'm not mistaken. I'm not familiar with field3d, so I don't know whether or not a rebuild would have caused field3d to pick up libhdf5.200.dylib.

[eborisch (Eric A. Borisch)]

OpenImageIO should advertise its dependence on hdf5 so that it can get caught in the (semi-manual) revbump cycle whenever hdf5 is updated.

[jasonliu-- (Jason Liu)]

Originally I thought so too, but I grepped through the OpenImageIO source code, and its code never actually makes any calls to hdf5. It only ever calls field3d.

[jmroot (Joshua Root)]

Then OpenImageIO is overlinked, and that should be fixed.

[jasonliu-- (Jason Liu)]

That's what I find odd. In one of the CMakeLists.txt files, there's clearly the following line:

src/field3d.imageio/CMakeLists.txt:

add_oiio_plugin (field3dinput.cpp field3doutput.cpp
                 INCLUDE_DIRS ${FIELD3D_INCLUDES}
                 LINK_LIBRARIES ${FIELD3D_LIBRARY}
                                ${HDF5_LIBRARIES}
                                ${SZIP_LIBRARY})

But in the rest of OpenImageIO's source code, there are no other references to "hdf". I suppose that line could technically qualify oiio as being dependent on hdf5, as Eric had said.

[jmroot (Joshua Root)]

As things currently stand, openimageio needs its revision bumped whenever hdf5's library major version changes. From what you've said, it looks like that may not need to be the case though.

[Dave-Allured (Dave Allured)]

I'm partly responsible for the HDF5 update in ​https://github.com/macports/macports-ports/pull/7983, so I should pitch in here. It appears that the dependency field3d is happy with the update to HDF5 1.12.0. Also I agree with Joshua's initial assessment, this is overlinking and nothing more. Adding a secondary dependency would be counterproductive.

Now I see that OpenImageIO has undergone some major changes in the past 6 months since since the MacPorts current version 2.1.7.0. OIIO is currently at 2.2.11.1. Some of those changes are in the CMake system. I am no CMake expert, but I can experiment with the most recent OIIO version.

Meanwhile, has anyone else tried any recent release of OpenImageIO? Is there some known reason to hold back to older OIIO versions?

[jasonliu-- (Jason Liu)]

If OIIO has undergone major changes in the past 6 months, then it's quite possible that updating the OIIO port could break anything that's dependent on it. For example, the newest version of Blender (released on 2021-Jan-20) was written against OIIO 2.1.15.0. It's possible that Blender would still work if compiled against the latest OIIO, but I haven't had time to try it.

[Dave-Allured (Dave Allured)]

Reported upstream: ​https://devtalk.blender.org/t/openimageio-update-to-2-2-11-1-or-latest/17410

The first responder calls this OIIO update a "minor release" compared to 2.1.5.0, therefore probably safe to update here. I don't know about the handful of other ports that are dependent on OIIO.

[jasonliu-- (Jason Liu)]

I'm not so sure that when the person said "minor release", that they intended "minor" to mean the same thing as "insignificant". Going from 2.1.x to 2.2.x is indeed an increment to the "minor version number", if you go by the nomenclature of <major>.<minor>.<patch> for version numbering. However, reading through the release notes on OIIO Github, the things that were added or changed between OIIO the 2.1 and 2.2 branches is a pretty long list.

Since I am the person who wrote the portfiles for all of the ports that are dependent on OIIO, I would need to try building all of those ports against an updated OIIO portfile before I would be comfortable with having a new version of OIIO committed to the public ports tree.

Regardless, looking at the builders, it looks like the buildbots have "caught up" with having a working combination of hdf5 / field3d / openimageio, so the ​builds for Blender are currently working, for the time being.

I'm not saying that we shouldn't update OIIO to a newer version. It's just that I'm currently working on an update to the USD and Blender ports, so I'll need to circle back around to OIIO after I get those merged. If the newer versions of OIIO solve the overlinking problem with hdf5, then obviously that would be best. It is definitely an encouraging sign, though, if Arch's Blender package is currently being built against the newest OIIO 2.2 release.

[Dave-Allured (Dave Allured)]

Jason, I appreciate your efforts and level of care with the OIIO related ports. Please take your time, only update OIIO when you think it is appropriate. Thanks.

[catap (Kirill A. Korinsky)]

In 4493efc8478e2bc705529a3b2d3b87a259e39107/macports-ports (master):

field3d: fix build before macOS 10.9

Closes: #62065

[jmroot (Joshua Root)]

@catap: Is the OpenImageIO overlinking that is the real underlying bug here actually fixed?

[catap (Kirill A. Korinsky)]

Replying to jmroot:

@catap: Is the OpenImageIO overlinking that is the real underlying bug here actually fixed?

Seems so.