make/tests/scripts/features
Paul Smith 7ad2593b2d Support implementing the jobserver using named pipes
Using anonymous pipes for jobserver support has some advantages:
for example there is nothing on disk that needs to be cleaned up.
However it has many obscure problems, related to the fact that in
order for it to work we need to ensure these resources are properly
passed through to child processes that want to use the jobserver.
At the same time we don't want to pass the pipe to process which
DON'T know about the jobserver.

Other processes can open file descriptors which we then think are
our jobserver, but aren't.  And, we open the pipe file descriptors
in blocking mode which doesn't work for all users.

See issues such as SV 57178, SV 57242, and SV 62397

To avoid these issues, use named pipes (on systems where they are
available) instead of anonoymous pipes.  This simplifies many things:
we never need to pass open file descriptors to our children; they
can open the jobserver named pipe.  We don't need to worry about
recursive vs. non-recursive children.  Users don't have to "pass
through" the resources if they are invoking sub-makes.  Each child
can open its own file descriptor and set blocking as needed.

The downside is the named pipe exists on disk and so must be cleaned
up when the "top-level" make instance exits.

In order to allow make to continue to be used in build systems where
older versions of GNU make, or other tools that want to use the
jobserver, but don't understand named pipes, introduce a new option
--jobserver-style that allows the user to choose anonymous pipes.

* NEWS: Announce the change and the --jobserver-style option.
* doc/make.1: Add --jobserver-style documentation.
* doc/make.texi (Special Variables): Add missing items to .FEATURES.
(Options Summary): Add --jobserver-style.
(POSIX Jobserver): Named pipes, changes to --jobserver-auth, and the
--jobserver-style option.
(Windows Jobserver): Document --jobserver-style for Windows.
* configure.ac: Check for mkfifo.
* src/config.h-vms.template: Undefined HAVE_MKFIFO.
* src/config.h.W32.template: Ditto.
* src/main.c: Add jobserver-style as a new command line option.
(main): Add jobserver-fifo to .FEATURES if supported.  Pass the style
option to jobserver_setup().
* src/os.h (jobserver_setup): Accept a style string option.
* src/posixos.c (enum js_type): Enumeration of the jobserver style.
(js_type): Which style we are currently using.
(fifo_name): The path to the named pipe (if in use).
(jobserver_setup): If no style is given, or "fifo" is given, set up a
named pipe: get a temporary file and use mkfifo() on it, then open it
for reading and writing.  If something fails fall back to anonymous
pipes.
(jobserver_parse_auth): Parse jobserver-auth to determine the style.
If we are using a named pipe, open it.  If we're using anonymous pipes
ensure they're valid as before.
(jobserver_get_invalid_auth): Don't invalidate the jobserver when
using named pipes.
(jobserver_clear): Clean up memory used for named pipes.
(jobserver_acquire_all): Unlink the named pipe when done.
* src/w32/w32os.c (jobserver_setup): Check the style argument.
* tests/scripts/features/jobserver: Use --jobserver-style to test
the anonymous pipe behavior, and also test named pipe/semaphore
behavior.  Check invalid jobserver-style options.
* tests/scripts/functions/shell: Use --jobserver-style to test the
anonymous pipe behavior, and also test named pipe/semaphore
behavior.
2022-08-02 23:36:35 -04:00
..
archives * tests/scripts/features/archives: Fix typo 2022-06-04 19:04:37 -04:00
comments * Added the test suite to the main distribution. 1999-09-14 02:03:19 +00:00
conditionals * read.c (eval): [SV 47960] Skip record waiting files when ignoring. 2016-05-21 16:22:32 -04:00
default_names Clean up some tests for use on Windows. 2013-10-05 19:30:48 -04:00
dircache [SV 41273] Allow the directory cache to be invalidated 2020-11-29 17:59:16 -05:00
double_colon * tests/scripts/features/double_colon: Update for modern tests. 2022-02-06 18:46:32 -05:00
echoing Portability changes for the test suite. 2016-12-26 09:01:59 -05:00
errors * tests/scripts/features/errors: Fix Perl warnings 2022-04-18 16:57:06 -04:00
escape Add test suite support to Windows 2017-06-04 18:37:20 -04:00
exec * tests/scripts/features/exec: Small cleanup 2021-03-15 01:17:06 -04:00
export Support "unexport" in target-specific variables. 2020-11-29 17:57:33 -05:00
grouped_targets [SV 48643] Add more tests of intermediates and unrelated targets 2022-02-06 18:46:32 -05:00
implicit_search * tests/scripts/features/implicit_search: Add -r for safety. 2022-02-06 18:46:32 -05:00
include tests: Allow run_make_test() to avoid passing -f 2022-02-27 17:47:42 -05:00
jobserver Support implementing the jobserver using named pipes 2022-08-02 23:36:35 -04:00
load tests: Convert %CONFIG_FLAGS to get_config() 2019-10-05 17:47:52 -04:00
loadapi tests: Convert %CONFIG_FLAGS to get_config() 2019-10-05 17:47:52 -04:00
mult_rules [SV 62650] Show correct warning when combining targets 2022-07-09 10:44:00 -04:00
mult_targets * Added the test suite to the main distribution. 1999-09-14 02:03:19 +00:00
order_only Make second expansion optional (partial implementation). 2005-10-24 13:01:39 +00:00
output-sync Rename jhelp.pl to thelp.pl and make it a generic test helper. 2019-12-16 09:31:31 -05:00
override - Modify access of config and gnulib Savannah modules to use GIT 2009-06-04 06:30:27 +00:00
parallelism tests: Simplify customization of %ENV 2020-04-01 02:02:57 -04:00
patspecific_vars [SV 39709] Fix some typos. 2013-10-09 01:53:55 -04:00
patternrules [SV 62206] Fix %-substitution in second expansion of pattern rules 2022-04-24 17:52:07 -04:00
quoting Add test suite support to Windows 2017-06-04 18:37:20 -04:00
recursion [SV 46013] Allow recursive variable overrides from Makefiles 2019-05-19 19:25:50 -04:00
reinvoke * tests/scripts/features/reinvoke: [SV 62088] Close STDIN 2022-04-24 13:48:03 -04:00
rule_glob [Bug #39310] Parse simple pattern prereqs for globbing. 2013-07-22 02:23:02 -04:00
se_explicit [SV 62706] Only second-expand targets that might be built 2022-07-30 18:40:28 -04:00
se_implicit [SV 62706] Only second-expand targets that might be built 2022-07-30 18:40:28 -04:00
se_statpat [SV 62706] Only second-expand targets that might be built 2022-07-30 18:40:28 -04:00
shell_assignment Add new feature: != shell assignment for portability with BSD make. 2011-04-18 01:25:20 +00:00
statipattrules * tests/scripts/features/temp_stdin: Remove nested "my" 2022-06-04 19:04:37 -04:00
suffixrules [SV 40657] Reinstate old behavior for suffix rules with prereqs 2020-01-19 17:04:52 -05:00
targetvars Special-case Windows error results 2021-11-28 14:27:10 -05:00
temp_stdin * tests/scripts/features/temp_stdin: Remove nested "my" 2022-06-04 19:04:37 -04:00
utf8 Ignore UTF-8 BOMs. See Savannah bug #36529. 2012-09-09 21:11:00 +00:00
varnesting Save strings we're expanding in case an embedded eval causes them 2011-08-29 16:20:19 +00:00
vpath Convert all "`'" quotes to "''" per new GNU Coding Standard guidelines. 2012-03-04 00:24:20 +00:00
vpath2 Refresh the test suite framework implementation. 2019-09-16 08:25:33 -04:00
vpath3 [SV 41758]: Fix archive support for VMS. 2014-10-20 01:31:42 -04:00
vpathgpath * tests/scripts/features/vpathgpath: Avoid duplicate function name. 2020-11-29 17:59:40 -05:00
vpathplus * tests/scripts/features/vpathplus: Fix output for big-endian systems. 2019-09-22 17:35:51 -04:00