Savannah issues such as SV 57242 and SV 62397 show how passing
references to closed file descriptors via the --jobserver-auth option
in MAKEFLAGS can lead to problematic outcomes.
When computing the child environment for a non-recursive shell, add
an extra option to MAKEFLAGS to disable the file descriptors for the
jobserver.
Unfortunately this doesn't modify the value of the make variable
MAKEFLAGS, it only modifies the value of the sub-shell environment
variable MAKEFLAGS. This can lead to confusion if the user is not
considering the distinction.
* src/makeint.h: Publish the jobserver-auth value. Add a global
definition of the name of the command line option.
* src/os.h (jobserver_get_invalid_auth): New function to return a
string invalidating the jobserver-auth option.
* src/w32/w32os.c (jobserver_get_invaid_auth): Implement it. On
Windows we use a semaphore so there's no need to invalidate.
* src/posixos.c (jobserver_parse_auth): If we parse the invalid
auth string, don't set up the jobserver.
(jobserver_get_invalid_auth): Return an invalid option.
* src/variable.h (target_environment): Specify if the target
environment is for a recursive shell or non-recursive shell.
* src/variable.c (target_environment): Move checking for MAKELEVEL
into the loop rather than doing it at the end.
Along with this, check for MAKEFLAGS and MFLAGS, and update them
based on whether we're invoking a recursive or non-recursive child,
and also on whether it's necessary to invalidate the jobserver.
* src/function.c (func_shell_base): Shell functions can never be
recursive to pass 0 to target_environment().
* src/job.c (start_job_command): Specify whether the child is
recursive when calling target_environment().
* src/main.c: Export jobserver_auth. sync_mutex doesn't need to
be exported. Use the global definition for the option name.
* tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
* src/makeint.h (stpcpy): Add missing declaration.
* src/amiga.c (MyExecute): Avoid extra strlen using stpcpy.
* src/function.c (func_shell_base): Ditto.
(func_error): Use memcpy() not strcpy() when we know the length.
* src/job.c (construct_command_argv_internal): Use stpcpy().
* src/main.c (main): Ditto.
(define_makeflags): Ditto.
* src/variable.c (print_target_variables): Use memcpy() when we know
the length.
* src/commands.c (set_file_variables): Use mempcpy().
* src/expand.c (variable_buffer_output): Ditto.
* src/file.c (expand_deps): Ditto.
* src/function.c (abspath): Ditto.
(handle_function): Ditto.
* src/implicit.c (pattern_search): Ditto.
* src/job.c (construct_command_argv_internal): Use mempcpy() and
don't add multiple spaces when there are no shell flags.
* src/main.c (decode_env_switches): Use mempcpy() to simplify.
(define_makeflags): Ditto.
* src/variable.c (selective_vpath_search): Ditto.
Export all variables, including exported makefile variables, when
invoking a shell for the $(shell ...) function. If we detect a
recursive variable expansion, silently ignore that variable and do
not export it. We do print a debug message.
* NEWS: Announce the potential backward-incompatibility.
* doc/make.texi (Shell Function): Document the export behavior.
* src/main.c (main): Add "shell-export" to .FEATURES.
* src/job.h: New function to free struct childbase.
* src/job.c (free_childbase): Implement it; call from free_child.
* src/function.c (func_shell_base): Use target_environment() to
obtain the proper environment for the shell function.
Use free_childbase() to free memory.
(windows32_openpipe): Don't reset the environment: the caller
already provided a proper PATH variable in envp.
* src/variable.c (target_environment): If we detect a recursive
expansion and we're called from func_shell, ignore the variable.
(sync_Path_environment): Simplify and reduce memory allocation.
* tests/scripts/functions/shell: Add tests for this.
memcmp() is always faster than strcmp(). In places where we already
know that both buffers have sufficient size, replace strcmp() with
memcmp().
* src/main.c (main): Replace strncmp() with memcmp()==0.
* src/read.c (word1eq): Ditto.
* src/commands.c (set_file_variables): Ditto.
* src/function.c (func_filter_filterout): Ditto.
(a_word_hash_cmp): Use STRING_N_COMPARE since we know the length.
(func_sort): Replace strcmp() with memcmp().
If a $(shell ...) invocation failed due to a command-not-found error,
make wrote the stdout of that shell to our stderr for some reason.
That seems very wrong.
If the command's stderr was not redirected then its output would have
already been written to its stderr, and if it was redirected then we
shouldn't take it upon ourselves to force it to go to stderr!
* src/function.c (func_shell_base): Append shell stdout even if the
shell command failed.
* tests/run_make_tests.pl: Determine the error generated for
command-not-found situations.
* tests/scripts/functions/shell: Verify that redirecting stderr to
stdout will behave properly if the command is not found.
Since $(info ) function output is written using two system calls for
the message and the newline, it's possible for output from another
parallel make job to sneak in between them.
Reported by Paul Eggert <eggert@cs.ucla.edu>, who saw the report from
Lars Ingebrigtsen <https://bugs.gnu.org/53358>.
* src/function.c (func_error): Paul provided a fix but instead I
rewrote the entire function: it's not possible for it to be invoked
with anything other than exactly one argument so don't worry about
re-combining the arguments.
Since we don't know what a loaded function (via Guile or load) may
do, increment the command count just in case.
* src/function.c (struct file_table_entry): New adds_command bool.
(FT_ENTRY): Initialize it to 0 for built-in functions.
(expand_builtin_function): If adds_command, increment the count.
(define_new_function): Set adds_command for loaded functions.
We don't need to parse strings into C integer values to compare them.
* src/function.c (parse_textint): Find boundaries of a numeric string.
(func_intcmp): Use parse_textint() to compare integers textually.
* tests/scripts/functions/intcmp: Test with extra-large numbers.
* src/function.c (parse_numeric): Check for empty value and error.
If we find ERANGE just print our own error, not strerror.
(func_word): Use a generic "not good" error message.
(func_wordlist): Ditto
(func_intcmp): Ditto
* tests/run_make_tests.pl: Remove code to find strerror(ERANGE)
* tests/scrips/functions/intcmp: Update the error message.
* tests/scrips/functions/word: Ditto.
Modify make functions that parse integer values to use long long
values instead of long: on Windows long is the same as int (4 bytes)
and we don't want behavior to differ between different platforms.
* bootstrap.conf: Change strtol module to strtoll module.
* src/function.c (parse_numeric): Use strtoll() and return long long.
(func_word): Use long long.
(func_wordlist): Use long long. Verify second argument is >= 0.
(func_intcmp): Use long long.
* src/config.ami.template: Don't define HAVE_STRTOLL.
* src/config-vms.template: Define HAVE_STRTOLL.
* src/config.W32.template: Define HAVE_STRTOLL.
* tests/run_make_tests.pl: Set $ERR_out_of_range to the proper string.
* tests/scripts/functions/word: Rework to use the new style and avoid
TAB characters. Verify trailing whitespace is ignored. Add a test
for a negative second argument to wordlist. Add tests for max signed
integer values. Use $ERR_out_of_range for the error string.
* tests/scripts/functions/intcmp: Add tests for min and max signed
integer values. Use $ERR_out_of_range for the error string.
Numbers can come from $(words ...), automatic variables such as
$(MAKELEVEL), from environment variables, or from shell output such as
through $(shell expr ...). The $(intcmp ...) function allows
conditional evaluation controlled by numerical variables.
* NEWS: Announce this feature.
* doc/make.texi (Functions for Conditionals): Document 'intcmp'.
* src/function.c (func_intcmp): Create the 'intcmp' built-in function.
* tests/scripts/functions/intcmp: Test the 'intcmp' built-in function.
strtol() is part of C89 and a fallback is provided by gnulib.
* src/function.c (func_word, func_wordlist): Change atoi to strtol.
* test/scripts/functions/word: Add out-of-range verification testing.
POSIX Issue 8 will require a new assignment operator, :::=.
This operator behaves similarly to the BSD make := operator: the
right-hand side is expanded immediately, but then the value is
re-escaped (all '$' are converted to '$$') and the resulting variable
is considered a recursive variable: the value is re-expanded on use.
* src/variable.h (enum variable_flavor): Add f_expand flavor.
* src/variable.c (do_variable_definition): When defining f_expand,
post-process the result to re-escape '$' characters.
Remove default: to the compiler warns about un-handled enum values.
Set recursive values for both f_recursive and f_expand.
(parse_variable_definition): Rewrite this method.
The previous version was annoying to extend to ':::='.
(print_variable): Remove default: so the compiler warns us about
un-handled enum values.
* src/function.c (func_origin): Remove default: so the compiler warns
us about un-handled enum values.
* doc/make.texi: Add documentation for :::=.
* tests/scripts/variables/define: Add a test for define :::=.
* tests/scripts/variables/flavors: Add tests for :::=.
* tests/scripts/variables/negative: Add tests for :::=.
Keep a count of bytes read rather than comparing pointers since the
variable_buffer might get reallocated.
Bug and patch by Ken Tossell <ken@tossell.net>
Regression tests by Dmitry Goncharov <dgoncharov@users.sf.net>
Tweaked by Paul Smith <psmith@gnu.org>
* src/function.c (func_file): Use bytes read rather than a pointer.
* tests/scripts/functions/file: Provide various tests for reading
empty files, files with/without newlines, and large files.
Add a new function $(let ...) which allows lexically scoped variables.
* NEWS: Add information on this feature.
* doc/make.texi (Let Function): Document the 'let' function.
* src/function.c (func_let): Create the 'let' built-in function.
* tests/scripts/functions/let: Test the 'let' built-in function.
Each time we invoke a command it's possible that it will change the
filesystem in ways that were not described by the target. If that
happens but we have cached previous directory contents then we may
make decisions or report results based on obsolete information.
Keep a count of how many commands we've invoked, and remember the
current command count every time we load the contents of a directory.
If we request the directory and the current command count has changed
we know the cache is outdated so reload from scratch.
* NEWS: Announce the change.
* src/makeint.h (command_count): Create a global counter.
* src/main.c (command_count): Ditto.
* src/job.c (reap_children): Increment the counter on job completion.
* src/function.c (func_file): Increment if we write a file.
* src/dir.c (clear_directory_contents): Clear the current contents of
a cached directory.
(struct directory_contents): Remember the counter value.
(struct directory): Remember the counter value for non-existing dirs.
(find_directory): If we have a cached directory and the count hasn't
changed then return it. Else, clear the previous contents and re-read
from scratch.
* tests/scripts/features/dircache: Add tests of the directory cache.
* src/function.c (func_filter_filterout): Allocate arrays to hold
pattern and word information rather than creating linked lists on
the stack.
* tests/scripts/functions/filter-out: Test large filters.
Create a struct childbase which is the basics of struct child needed
to invoke child_execute_job(), and can be cast back and forth to a
struct child. Ensure all implementations of child_execute_job() take
the same arguments.
* src/job.h (CHILDBASE): Create a macro to hold the base parts.
(struct childbase): A structure containing CHILDBASE.
(struct child): Use CHILDBASE as the initial part of child.
[VMS]: Remove declaration of VMS version of child_execute_job.
* src/job.c (start_job_command): Use new child_execute_job() call.
(child_execute_job) [__EMX__]: Implement new calling signature.
(child_execute_job): Implement new calling signature.
* src/main.c (main) [__EMX__]: Use new child_execute_job() call.
* src/function.c (func_shell_base): Use new child_execute_job() call.
* src/vmsjobs.c (vmsHandleChildTerm): Accept struct childbase.
* src/vmsjobs.c (child_execute_job): Implement new calling signature.
Modify the return value to be pid_t, not a boolean, and return the
PID rather than setting it in the child. This is OK because our only
caller immediately reset PID to -1 anyway if we return 0.
* src/function.c (func_shell_base): Use error() instead of recreating
the error output.
* src/job.c (exec_command): Show more standard error messages.
* src/load.c (unload_file): Fix whitespace in the error message.
* tests/scripts/features/errors: Add tests for starting non-
existent commands and new error message formats.
* tests/scripts/features/output-sync: New error message formats.
* tests/scripts/functions/shell: Ditto.
If we failed to fork() we were essentially exiting make immediately
without respect to ignore flags, etc. On one hand that makes sense
because if you can't fork you're in real trouble, but it doesn't
work so well on systems where we don't fork at all. Instead, treat
a fork error like any other error by delaying the handling until
the next call to reap_children(). Any child with a PID of -1 is
considered to have died before starting so check these first without
waiting for them.
* src/commands.c (fatal_error_signal): Don't kill children that
never started.
* src/function.c (func_shell_base): Handle cleanup properly if the
child doesn't start.
* src/job.c (reap_children): Check for children that died before
starting and handle them without waiting for the PID.
(start_job_command): Free memory when the child doesn't start.
(start_waiting_job): Don't manage children who never started.
(child_execute_job): If the fork fails return PID -1.
* src/vmsjobs.c: Check for children that never started.
* tests/run_make_tests.pl: Parse config.status to get all options.
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.
* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
