Repository navigation
should libgap be fork(2)-safe? #6456
Description
We have a scenario where an application which uses libgap calls fork on itself. It appears to work, except in the case where the corresponding installation of GAP has Browse package installed. In this case fork call leads to a lock-up, involving libgap calling ncurses.so GAP kernel module from Browse, which in turns calls libncurses or its equvalent, and, oops, it all freezes. Hitting Ctrl-C produces a dump showing that just mentioned components involved. See e.g. gap-system/homebrew-gap#23 (comment)
As there is no Unload equivalent to LoadPackage, the most obvious workaround is not having Browse installed. Which is kind of silly. Should work be put into getting it fixed?
Activity
ChrisJefferson commentedon Jul 8, 2026
I think this is more complex than 'libgap is fork safe', I mean, unless there is a specific bug in GAP, I've auto-loaded Browse and used IO_fork from the IO package for years, many people have, and everything works fine. Can you produce a minimal example which just uses libgap (not the entirety of sage) and show the problem? I suppose we might be somehow specially catching browse some how, but I really don't want to debug this via sage.
dimpase commentedon Jul 8, 2026
The problem comes from using python3 multiprocessing with
forkoption on python code with a libgap extension. The quickest way to have a minimal reproducer might be to write it in C, something I'm not quite looking forward to - hopefully it's not needed.I understand that Browse does have code testing whether it outputs to tty, and (probably?) just doesn't do anything if this is not the case, right? So perhaps the question is about setting the appropriate environment for Browse to get this across. Do you know whether IO_fork does something non-obvious to deal with this?
dimpase commentedon Jul 9, 2026
OK, Sage shoots itself in the foot by disabling SIGCHLD signal in its libgap initialization code:
https://github.com/sagemath/sage/blob/80e56eaeb7089cfc5e4e0cc495ba8da43ee0a573/src/sage/libs/gap/util.pyx#L343
It makes it more complicated that also cysignals is involved in the whole thing. @orlitzky - you touched libgap calling recently...
Also, the docs of
IO_InstallSIGCHLDHandlerare a bit cryptic, sayingWhat's any function here? Does it cover calling fork(2) on a process which loaded libgap/IO?
fingolfin commentedon Jul 16, 2026
"Any function" here refers of course to any function defined by the IO package, i.e., the
IO_*functions such asIO_fork, etc.If you search the documentation for
IO_InstallSIGCHLDHandleryou'll also see it being mentioned explicitly in e.g.IO_Popen,IO_Popen2etc.ChrisJefferson commentedon Aug 17, 2026
So for anyone reading this from Sage, GAP's handling of children is, to be honest, a little messy (and how IO and GAP core interact on this is wobbly as well), but it's very hard to tidy up due to not wanting to break backwards compatibility.
dimpase commentedon Oct 17, 2026
I now have a small C/libgap reproducer for the Browse symptom. It does not
use Sage, Python, cysignals, or IO. It initializes libgap, forks, puts the
child in its own process group, then loads Browse with
OnlyNeeded := true.A tiny Python harness only supplies a controlling tty and execs the C test.
On macOS 27.0.1 arm64, GAP 4.16.1 and Browse 1.8.24 (released 2026-10-08):
The parent detects the stop with
waitpid(..., WUNTRACED)and kills/reapsthe test child. The current release was tested with Homebrew ncurses 6;
earlier tests with Browse 1.8.23 reproduced the stop using both Apple's
ncurses 5.4 and Homebrew's ncurses 6. Thus the earlier SIGCHLD explanation
does not account for this reproducer. Browse's unconditional
initscr()inPostRestorechanges terminal modes from a background process, matching the original
tcsetattr/cbreak/newtermtraceback. Sage's doctest workers callsetpgid, so this reproduces a relevant part of their environment.Browse 1.8.24 already restores SIGWINCH, SIGTSTP and SIGCONT handlers after
ncurses initialization to address Julia/Oscar conflicts. That does not
prevent the SIGTTOU stop during initialization observed here.
There is a small opt-in workaround in the proposed patch: make Browse's
existing
loadwithoutncursespreference effective even when its kernelmodule is installed. It selects the existing nonvisual fallback. The patch
also removes an existing mutation of the registered package name which
otherwise breaks fallback loading with
OnlyNeeded := true.With that preference set before the first Browse load, the background C
test stops before the patch and succeeds after it, including a nonvisual
Browse operation. Ordinary foreground loading still succeeds. This does
not solve general background interactive use or restoration of an existing
workspace containing the ncurses module. Deferring terminal initialization
until interactive use would be the broader Browse fix.
Two separate SIGCHLD findings:
Make IO_fork children stoppable when GAP is embedded gap-packages/io#137 and
Reap only own children when GAP is embedded or HPC-GAP gap-packages/io#138. A separate C test shows IO
4.10 replacing the host handler and consuming a host child's exit status;
the exact v4.11.0 release preserves both.
GAP_Initialize(..., handleSignals=0), as tracked in Allow disabling of GAP's SIGCHLD handler #3380. Theunconditional installation also remains in current master. Its current
handler waits for tracked pty children, not arbitrary children.
The same runner has now also been run on Gentoo x86_64 Linux 7.0.3,
with glibc 2.43, GAP/libgap 4.16.1, Browse 1.8.24 and ncurses/panel
6.5.20251220. Its seven before-patch and three after-patch outcomes match
the macOS results exactly, with no supervisor timeouts. The nonvisual GAP
assertions pass too. Linkage logs confirm /usr/lib64/libgap.so.11 and the
system ncurses/panel libraries. The separate core-handler and IO 4.10
child-ownership failures also reproduce. IO 4.11 was tested only on macOS,
not in this Linux run.
These tests isolate particular failures; they do not establish general
post-fork safety or validate the full interactive Browse functionality.
AI disclosure: OpenAI Codex assisted with the investigation, reproducers,
proposed Browse patch, tests, and this report.
The C reproducers, test runner, proposed Browse patch, and recorded results
will follow as a separate archive attachment.
dimpase commentedon Oct 17, 2026
gap-6456-sources.zip