ocaml: fails to build on 10.6 and lower

[mojca (Mojca Miklavec)]

OCaml port fails to build on 10.6 and lower. It might help to use a newer compiler, but it makes sense to test the hypothesis before blindly adding items to compilers blacklist.

/usr/bin/gcc-4.2 -c -DSYS_macosx -DMODEL_default -o amd64.o amd64.S || \
	{ echo "If your assembler produced syntax errors, it is probably";\
          echo "unhappy with the preprocessor. Check your assembler, or";\
          echo "try producing amd64.o by hand.";\
          exit 2; }
amd64.S:708:Unknown pseudo-op: .string
amd64.S:708:Rest of line ignored. 1st junk character valued 34 (").
If your assembler produced syntax errors, it is probably
unhappy with the preprocessor. Check your assembler, or
try producing amd64.o by hand.

If it turns out that it in fact cannot be built, the port should at least say so. But I guess it's just an issue with too old compiler.

On 10.5 it failed with

/usr/bin/gcc-4.2 -bundle -flat_namespace -undefined suppress                    -Wl,-no_compact_unwind -o libcamlrun_shared.so interp.pic.o misc.pic.o stacks.pic.o fix_code.pic.o startup_aux.pic.o startup.pic.o freelist.pic.o major_gc.pic.o minor_gc.pic.o memory.pic.o alloc.pic.o roots.pic.o globroots.pic.o fail.pic.o signals.pic.o signals_byt.pic.o printexc.pic.o backtrace_prim.pic.o backtrace.pic.o compare.pic.o ints.pic.o floats.pic.o str.pic.o array.pic.o io.pic.o extern.pic.o intern.pic.o hash.pic.o sys.pic.o meta.pic.o parsing.pic.o gc_ctrl.pic.o terminfo.pic.o md5.pic.o obj.pic.o lexing.pic.o callback.pic.o debugger.pic.o weak.pic.o compact.pic.o finalise.pic.o custom.pic.o dynlink.pic.o spacetime.pic.o afl.pic.o unix.pic.o main.pic.o -lcurses -lpthread                  
ld: unknown option: -no_compact_unwind

• ​10.5
• ​10.6/i386
• ​10.6/x86_64

[mojca (Mojca Miklavec)]

Another totally weird failure to build ​coq.

[pmetzger (Perry E. Metzger)]

My guess is that indeed we need to depend (on those older platforms only) on a newer C compiler.

I lack direct access to a 10.6 or 10.5 machine, so it may be hard for me to do tests without blind changes.

How many users do those old versions have btw? I presume it is considered important to fix them?

For the 10.10 failure, am I reading the log correctly? Did something kill the build with -9? Maybe it isn't a real build failure but was rather a one time oddity of some sort?

[mojca (Mojca Miklavec)]

I have no numbers, but 10.6 is still somehow popular for enthusiasts. Maintainers are not expected to fix the problems if they don't want to, it purely depends on the enthusiasm of each individual. If it's just a matter of blacklisting gcc 4.2, it might be worth a shot. If it requires heavy patching, there's no need to waste your precious time.

For coq on 10.10 check ​this build (once ready).

[kencu (Ken)]

I can check easily. I'll get back to you.

10.6 is a "sticky" system version, with adherents like me. There are lots of reasons -- Apple just hit a sweet spot for many of us with that version.

[pmetzger (Perry E. Metzger)]

Please do check if it can be fixed just by altering the compiler. A patch for the Portfile that's conditioned on OS X <= 10.6 would be appreciated too.

[kencu (Ken)]

My default 10.6.8 system is set up with LibcxxOnOlderSystems, using libc++, and the default compiler is set to clang-3.9. On that system (which I recommend for everyone to use, and dearly wish we had a buildbot for), ocaml built straight through without touching anything:

$ port -v installed ocaml
The following ports are currently installed:
  ocaml @4.02.2_1 (active) platform='darwin 10' archs='x86_64' date='2017-11-22T09:22:43-0800'

I will explore exactly what works, but in my mind, you should either let the default compiler do it, or jump directly to a new clang (like 4 or 5). I personally think it's kind of tedious to spend too much time finding the oldest compiler that works (a moving target for the next version of the port anyway, usually).

[mojca (Mojca Miklavec)]

Repeated attempt of build coq was successful.

Let's give blacklisting gcc 4.0 and 4.2 one shot. If that doesn't help, we can figure out the details once again.

[mojca (Mojca Miklavec)]

In ccfdeabbecfe8d97d1e888eefea91b4c85f0525b/macports-ports:

ocaml: blacklist gcc 4.0 and 4.2

See: #55385

[mojca (Mojca Miklavec)]

OK, for ​10.5 this was a failed experiment. The problem existed before and doesn't seem to be connected with the compiler in any way (sorry, I'll help you remove the blacklisting :). See the existing ticket #40956 with claims that the problem was solved after patching the Makefiles.

​10.6/i386 using clang fails with

ld: absolute addressing (perhaps -mdynamic-no-pic) used in _caml_system__code_begin from i386.pic.o not allowed in slidable image. Use '-read_only_relocs suppress' to enable text relocs
clang: error: linker command failed with exit code 1 (use -v to see invocation)

​10.6/i386 using clang-3.4 fails with

ld: illegal text-relocoation (direct reference) to (global,weak) _caml_gc_regs in roots.pic.o from _caml_call_gc in i386.pic.o for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)

​10.6/x86_64 using clang ​10.6/x86_64 using clang-3.4

[mojca (Mojca Miklavec)]

I'm stealing this ticket for a while for other issues, but my suggestion would be the following:

• In case it's possible to trivially fix this particular issue first, that would be helpful because of the planned changes on 100+ other ports. (Btw: where did you get the number 300? Did you also count the patchfiles?) That is: if you move the other ports before fixing this, all those changes will generate errors on the buildbot just because of the failure to build dependencies and you would need to rebuild them later. But don't spend too much time on it as dealing with old systems is not the highest priority, it makes more sense to first make sure that ocaml works well on newer systems. (If you could remove that flag by some conditional patching on < 10.6, fine, else don't bother.)

• You may go through some old tickets (some are 10 years old) where the port field contains ocaml and figure out if any of them happen to be addressed already, or if an upgrade to the latest version solves the problem. If that requires small independent patches, you can submit them separately, so that we apply them before the "big move". If ports are obsolete, feel free to delete them. If some issues have been addressed, ask on IRC if someone can close them for you (or collect such requests in another ticket perhaps?)

• I would suggest to then try to move ports to a different subfolder and add an additional category. And let the buildbots try to build all of them. Just moving the ports should be sufficient to trigger a rebuild. You may collect some basic statistics about which ports need an updated etc. along the way.

• Then collect statistics about which ports failed, so that you can address those later.

Just a few ideas.

[pmetzger (Perry E. Metzger)]

So just to be clear, what would everyone recommend we do at this point? (I'd like to get moving with further downstream changes.)

[mojca (Mojca Miklavec)]

Just don't get swamped into solving this particular issue. I'll try another blacklisting for 10.6 and if that doesn't work, I'll leave it to others to figure it out.

My suggestion would be to either look at some existing tickets (if they don't require too much extra effort) or else concentrate into upgrading/fixing some highest priority ocaml-based ports, or perhaps start by moving the ports & adding the category with some assistance of scripting. Just do whatever will make you most productive and submit pull request(s).

If you don't plan to move the ports just yet, you can also send me a list of ports, so that we can quickly check on the buildbot which ones are broken. If you do plan to move them, that could be one of the first steps that will get you the builds as a side product.

[mojca (Mojca Miklavec)]

In 20caa81a96ce015ff8b1be8bff27f1b17532ba89/macports-ports:

ocaml: blacklist clang from 10.6

See: #55385

[kencu (Ken)]

-Wl,-read_only_relocs,suppress should probably be in base for all PPC and i386 builds. It is often needed. I think it used to be in base once upon a time.

[pmetzger (Perry E. Metzger)]

kencu, could you supply a patch after verifying that it works?

[kencu (Ken)]

will do. I guess you guys are in a bit of a hurry to get this going. Sorry I'm somewhat tied up in my day job just now.

[pmetzger (Perry E. Metzger)]

I have a huge number of OCaml ports to deal with so getting the number of open problems down is a priority. :(

I would try fixing 10.5 and 10.6 myself but I have no machines I can test on.

[kencu (Ken)]

I am working on this right now. I'm sorry I made an error above, and I built a slightly older ocaml version above. The current version that we're interested in, 4.0.5 fails to build due to a missing strnlen function.

../boot/ocamlrun ../ocamlc -nostdlib -I ../stdlib -linkall -I ../otherlibs/unix -o ocamldebug -linkall ../otherlibs/unix/unix.cma ../utils/config.cmo ../utils/tbl.cmo ../utils/misc.cmo ../utils/identifiable.cmo ../utils/numbers.cmo ../utils/arg_helper.cmo ../utils/clflags.cmo ../utils/consistbl.cmo ../utils/warnings.cmo ../utils/terminfo.cmo ../parsing/location.cmo ../parsing/longident.cmo ../parsing/docstrings.cmo ../parsing/syntaxerr.cmo ../parsing/ast_helper.cmo ../parsing/ast_mapper.cmo ../parsing/ast_iterator.cmo ../parsing/attr_helper.cmo ../parsing/builtin_attributes.cmo ../typing/ident.cmo ../typing/path.cmo ../typing/types.cmo ../typing/btype.cmo ../typing/primitive.cmo ../typing/typedtree.cmo ../typing/subst.cmo ../typing/predef.cmo ../typing/datarepr.cmo ../typing/cmi_format.cmo ../typing/env.cmo ../typing/oprint.cmo ../typing/ctype.cmo ../typing/printtyp.cmo ../typing/mtype.cmo ../typing/envaux.cmo ../bytecomp/runtimedef.cmo ../bytecomp/bytesections.cmo ../bytecomp/dll.cmo ../bytecomp/meta.cmo ../bytecomp/symtable.cmo ../bytecomp/opcodes.cmo ../driver/compdynlink.cmo ../toplevel/genprintval.cmo int64ops.cmo primitives.cmo unix_tools.cmo debugger_config.cmo parameters.cmo lexer.cmo input_handling.cmo question.cmo debugcom.cmo exec.cmo source.cmo pos.cmo checkpoints.cmo events.cmo program_loading.cmo symbols.cmo breakpoints.cmo trap_barrier.cmo history.cmo printval.cmo show_source.cmo time_travel.cmo program_management.cmo frames.cmo eval.cmo show_information.cmo loadprinter.cmo parser.cmo command_line.cmo main.cmo
File "_none_", line 1:
Error: Error on dynamically loaded library: ../otherlibs/unix/dllunix.so: dlopen(../otherlibs/unix/dllunix.so, 138): Symbol not found: _strnlen
  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_ocaml/ocaml/work/ocaml-4.05.0/otherlibs/unix/dllunix.so
  Expected in: flat namespace
 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_ocaml/ocaml/work/ocaml-4.05.0/otherlibs/unix/dllunix.so
make[3]: *** [ocamldebug] Error 2
make[2]: *** [ocamldebugger] Error 2
make[1]: *** [opt.opt] Error 2
make: *** [world.opt] Error 2
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_ocaml/ocaml/work/ocaml-4.05.0" && unset LD_PREBIND LD_PREBIND_ALLOW_OVERLAP && /usr/bin/make world.opt 

The snowleopard_fixes portgroup might normally fix this, but unfortunately it's somewhere in the ocaml build itself where it fails, not standard c++, so the configure.flags manipulations don't work.

Short story - at present it won't build on 10.6 or less, even with a new compiler. I can work on it, but it will take some time it appears. A patch for the missing function, maybe, or see how to link in the -lsnowleopardfixes library somehow in that link stage.

[mojca (Mojca Miklavec)]

Perry, for all I can say, don't worry about these issues and concentrate on more important ones for now. If this was a trivial fix with blacklisting a compiler or two, we could do it, but it makes no sense to put too much effort in this compared to the importance of cleaning up the ports for everyone else. You may try to submit a super low priority bug report upstream (but again, don't worry about it).

[kencu (Ken)]

Does anyone know how to add a link library to an ocaml build? I think all we need to do (AFAIK) is figure out how to get whatever ocaml sees as -lsnowleopardfixes.dylib added onto that build line that fails

../boot/ocamlrun ../ocamlc -nostdlib -I ../stdlib -linkall -I ../otherlibs/unix -o ocamldebug -linkall ../otherlibs/unix/unix.cma ../utils/config.cmo ../utils/tbl.cmo ../utils/misc.cmo ../utils/identifiable.cmo ../utils/numbers.cmo ../utils/arg_helper.cmo ../utils/clflags.cmo ../utils/consistbl.cmo ../utils/warnings.cmo ../utils/terminfo.cmo ../parsing/location.cmo ../parsing/longident.cmo ../parsing/docstrings.cmo ../parsing/syntaxerr.cmo ../parsing/ast_helper.cmo ../parsing/ast_mapper.cmo ../parsing/ast_iterator.cmo ../parsing/attr_helper.cmo ../parsing/builtin_attributes.cmo ../typing/ident.cmo ../typing/path.cmo ../typing/types.cmo ../typing/btype.cmo ../typing/primitive.cmo ../typing/typedtree.cmo ../typing/subst.cmo ../typing/predef.cmo ../typing/datarepr.cmo ../typing/cmi_format.cmo ../typing/env.cmo ../typing/oprint.cmo ../typing/ctype.cmo ../typing/printtyp.cmo ../typing/mtype.cmo ../typing/envaux.cmo ../bytecomp/runtimedef.cmo ../bytecomp/bytesections.cmo ../bytecomp/dll.cmo ../bytecomp/meta.cmo ../bytecomp/symtable.cmo ../bytecomp/opcodes.cmo ../driver/compdynlink.cmo ../toplevel/genprintval.cmo int64ops.cmo primitives.cmo unix_tools.cmo debugger_config.cmo parameters.cmo lexer.cmo input_handling.cmo question.cmo debugcom.cmo exec.cmo source.cmo pos.cmo checkpoints.cmo events.cmo program_loading.cmo symbols.cmo breakpoints.cmo trap_barrier.cmo history.cmo printval.cmo show_source.cmo time_travel.cmo program_management.cmo frames.cmo eval.cmo show_information.cmo loadprinter.cmo parser.cmo command_line.cmo main.cmo

and it should work. Maybe.

I just haven't a clue how to do that in ocaml-speak.

[kencu (Ken)]

fixed, at least for 10.6.8

$ port -v installed ocaml
The following ports are currently installed:
  ocaml @4.05.0_0 (active) platform='darwin 10' archs='x86_64' date='2017-12-15T14:13:57-0800'

[pmetzger (Perry E. Metzger)]

I see you've requested a pull on github. I've approved it, though the merge should wait until the CI system finishes its check.

[kencu (Ken)]

In 61940fe8470441051f7a77e62176e129d1e8e902/macports-ports:

ocaml: fix build error due to missing strnlen

fixes build on 10.6.8 and earlier
closes: #55385

[pmetzger (Perry E. Metzger)]

Reopened because the i386 and ppc builds remain broken for now.

[kencu (Ken)]

ocaml 4.05 build failure on PPC with gcc6

[kencu (Ken)]

PPC may not work out. Fails with clang-3.4 (of course), but fails with gcc6 as well. Cause of error not obvious to me.

[pmetzger (Perry E. Metzger)]

Does PPC work with the system compiler? If not, I'm inclined to mark the port only for i386 and x86-64. Thoughts?

[kencu (Ken)]

Yeah, the PPC build doesn't work with any compiler just yet. Looks like some kind of assembly code issue in the build script that I'm not inclined to bother with at present. If anyone ever cares about it, may help them push on it then. I wonder if any ocaml build ever worked on PPC.

Re: 32bit 10.6.8, I don't have a 10.6.8 i386 setup to test on, unfortunately. I think adding that linker flag I mentioned in the PR would probably fix that build. Only thing I can do is push a fix to the buildbot and see if it fails or not. Have to remember how to test for a 32bit build -- some kind of ${build.arch} test, I think.

[pmetzger (Perry E. Metzger)]

Try to fix the i386. I'm going to blacklist ppc.

[pmetzger (Perry E. Metzger)]

So Ken, it seems that ocamlbuild isn't building right on 10.6 because ocaml tries to invoke the same C compiler that it was built with, which isn't the default, and which is a bit hard to specify in port files. Not sure how to proceed from here.

[kencu (Ken)]

I think I know what you mean. We had the same issue with ghc, and had to fix this kind of problem there too by specifying a dependency on the build compiler. Can you git me a log or something to show how it fails?

[pmetzger (Perry E. Metzger)]

It's pretty mysterious, but look at:

​https://build.macports.org/builders/ports-10.6_i386_legacy-builder/builds/33965/steps/install-dependencies/logs/stdio

Things are going along quite smoothly and suddenly they blow up. I can't quite explain what's going on. The important bit is:

ocamlc.opt -w L -w R -w Z -I src -I +unix -safe-string -bin-annot -c src/ocamlbuild.mli
ocamlc.opt -w L -w R -w Z -I src -I +unix -safe-string -bin-annot -c src/ocamlbuild.ml
ocamlc.opt -I +unix -I src -o ocamlbuild.byte unix.cma src/ocamlbuild_pack.cmo src/ocamlbuild_plugin.cmo src/ocamlbuild_executor.cmo src/ocamlbuild_unix_plugin.cmo src/ocamlbuild.cmo
ocamlc.opt -a -o src/ocamlbuildlib.cma src/ocamlbuild_pack.cmo src/ocamlbuild_plugin.cmo src/ocamlbuild_executor.cmo src/ocamlbuild_unix_plugin.cmo
ocamlopt.opt -for-pack Ocamlbuild_pack -w L -w R -w Z -I src -I +unix -safe-string -bin-annot -c src/const.ml
sh: /opt/local/bin/clang-mp-3.4: No such file or directory
File "src/const.ml", line 1:
Error: Assembler error, input left in file /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_ocaml_ocaml-ocamlbuild/ocaml-ocamlbuild/work/.tmp/camlasm709cf1.s
make: *** [src/const.cmx] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_ocaml_ocaml-ocamlbuild/ocaml-ocamlbuild/work/ocamlbuild-0.12.0'
Command failed:  cd "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_ocaml_ocaml-ocamlbuild/ocaml-ocamlbuild/work/ocamlbuild-0.12.0" && /usr/bin/make -w all 
Exit code: 2

[kencu (Ken)]

Yep. Exactly the same error as ghc. This is the relevant part:

sh: /opt/local/bin/clang-mp-3.4: No such file or directory

ocaml will need a runtime dep on macports-clang-3.4 on 10.6.

We might as well do it all at once -- are there any other systems that fail?

[pmetzger (Perry E. Metzger)]

Both x86_64 and i386 on 10.6 errored out in the buildbot. I don't know if we support *86 on 10.5, I didn't see any errors from that?

ppc is broken on both 10.5 and 10.6 in the buildbot of course, but we knew that.

[kencu (Ken)]

10.5 Intel most likely works, so we'll make sure that is included in the test.

The 10.7 buildbot seems to be broken or offline -- I'll test it on my local VM.

10.8+ build OK with the default compiler?

[pmetzger (Perry E. Metzger)]

I'm pretty sure 10.7+ is okay with the default compiler, not just 10.8.

[pmetzger (Perry E. Metzger)]

BTW, for those reading this, I finally understood the difference in the transcript. (ocamlc.opt is a bytecode compiler, not a native compiler. ocamlopt.opt is the native compiler. It was dying the first time it invoked the native compiler.) So the failure is not so mysterious after all, I just had a reading comprehension failure.

[pmetzger (Perry E. Metzger)]

Looks like we have another fail

​https://build.macports.org/builders/ports-10.6_i386_legacy-watcher/builds/11338

I can't quite figure out why ocamlbuild isn't building now (too many files to sift through right now, but feel free...)

[kencu (Ken)]

ocaml-ocalmbuild built on 10.6.8 when I just force-rebuilt it:

<​https://build.macports.org/builders/ports-10.6_i386_legacy-builder/builds/34138/steps/install-port/logs/stdio>

It's a big, complicated mess of ports. We'll pick off the errors step by step.

[pmetzger (Perry E. Metzger)]

The errors in the build run I posted seemed to indicate ocaml-ocamlbuild hadn't rebuilt during that run and was the cause of most of the failures... any idea why the difference?

[kencu (Ken)]

Timing, probably. When that run you specified was run, the last attempt at ocaml-ocamlbuild had failed, and so the buildbot will not try to build it again unless something is done (either a forced rebuild attempt, or a change in the Portfile pushed to master).

[pmetzger (Perry E. Metzger)]

kencu, I'm kind of tempted to close this as "will not fix". Thoughts?

[kencu (Ken)]

Mmm. ocaml builds and runs fine for me on 10.6.8.

$ port -v installed | grep ocaml
  ocaml @4.05.0_0 (active) platform='darwin 10' archs='x86_64' date='2017-12-15T14:13:57-0800'

I think it more a buildbot issue. On the buildbot, ocaml builds with clang-3.4, and thereafter, requires clang-3.4 to run because it's hard-coded into the scripts.

But there was no run dep on clang-3.4, and so if someone installed ocaml and tried to use it, but clang-3.4 is not installed, ocaml failed.

Looks fixable to me. Is it still causing errors for people?

[pmetzger (Perry E. Metzger)]

It's certainly causing me headaches when the buildbot complains.

[barracuda156]

Replying to kencu:

PPC may not work out. Fails with clang-3.4 (of course), but fails with gcc6 as well. Cause of error not obvious to me.

PPC code was removed in 4.0.3, so I would be surprised if 4.0.5 would build without patching out the wrong Power code to replace it with the one borrowed from 4.0.2.

I am getting the same ld error now btw:

/opt/local/bin/gcc-mp-11 -arch ppc -shared                    -flat_namespace -undefined suppress -Wl,-no_compact_unwind -read_only_relocs suppress                    -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc -o libasmrun_shared.so startup_aux.npic.o startup_nat.npic.o main.npic.o fail_nat.npic.o roots_nat.npic.o signals.npic.o signals_nat.npic.o misc.npic.o freelist.npic.o major_gc.npic.o minor_gc.npic.o memory.npic.o alloc.npic.o compare.npic.o ints.npic.o floats.npic.o str.npic.o array.npic.o io.npic.o extern.npic.o intern.npic.o hash.npic.o sys.npic.o parsing.npic.o gc_ctrl.npic.o eventlog.npic.o md5.npic.o obj.npic.o lexing.npic.o unix.npic.o printexc.npic.o callback.npic.o weak.npic.o compact.npic.o finalise.npic.o custom.npic.o globroots.npic.o backtrace_nat.npic.o backtrace.npic.o dynlink_nat.npic.o debugger.npic.o meta.npic.o dynlink.npic.o clambda_checks.npic.o afl.npic.o bigarray.npic.o memprof.npic.o domain.npic.o skiplist.npic.o codefrag.npic.o power_libasmrunpic.o -lm 
ld: relocation used in _caml_system__code_begin from power_libasmrunpic.o not allowed in slidable image
collect2: error: ld returned 1 exit status
make[3]: *** [libasmrun_shared.so] Error 1
make[3]: Leaving directory `/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_ocaml/ocaml/work/ocaml-4.14.0/runtime'
make[2]: *** [makeruntimeopt] Error 2

[barracuda156]

Replying to pmetzger:

Looks like we have another fail

Apparently, I have fixed building of ocaml for PPC:

macmini:ocaml-4.14.0 svacchanda$ port -v installed ocaml
The following ports are currently installed:
  ocaml @4.14.0_0 (active) requested_variants='' platform='darwin 10' archs='ppc' date='2022-04-26T08:47:31+0800'

​https://github.com/macports/macports-ports/pull/14691