Commit Graph

198 Commits

Author SHA1 Message Date
Paul Smith
26e359c71d [SV 62840] Don't change IO buffering before printing version
If users run 'make --version | head -n1' they expect make to exit
with a success (0) code.  This works because the pipe forces the
default buffering on stdout to be fully buffered so all the output
is printed to the pipe in a single write(2) and won't fail.  However
due to output sync we forcibly set stdout to line buffered, which
means if the reader closes the pipe fast enough make will exit with
an error code because the write to stdout failed.

Move the setup done in output_init() back into main() where it can
be done in a proper order.  Rework the order of operations during
startup so that we check for help and version flags before we change
the buffering.  Clean up the behavior of print_usage().

Original changes from Dmitry Goncharov <dgoncharov@users.sf.net>.

* src/main.c (switches): Don't send --version in the environment.
(print_usage): Add a blank line after the version before the usage.
Move the die() into this function since we always die() afterward.
Note the die will flush so no need to do it explicitly.
(print_version): The caller will fflush when appropriate.
(close_stdout): Move from output.c so it is installed early.
(decode_switches): Only call print_usage on error, not for --help.
(main): Install the close_stdout handler immediately after start.
Move the calls to print_usage() due to --help and --version to be
called immediately after we decode the switches.  Move the buffer set
here from output_init(), immediately after we know we'll be running.
* src/output.c (output_init): Move buffer setting to main().
(close_stdout): Move to main().
2022-08-30 21:04:48 -04:00
Paul Smith
4da2055a10 Rework output sync to lock a temp file on POSIX
Some POSIX systems do not allow locks to be taken on non-files, such
as pipes.  This is a problem since very often make is invoked with
its stdout redirected to a pipe.  Also, if stdout is redirected to a
file that already has a lock on it for some other reason (perhaps a
shared file such as /dev/null) it can cause a hang.

This means our previous method of locking stdout, although it had some
nice advantages, is not portable enough.  Instead, use a temporary
file and take the lock on that.  We pass the name of the file to child
make processes.  On Windows we continue to use a shared mutex for
output sync.

Remove POSIX emulation functions like fcntl from Windows; instead
follow the lead of the jobserver and create an interface in os.h for
output sync, and move the OS-specific content to posixos.c and
w32os.c.

* NEWS: Add a note.
* src/makeint.h (ALL_SET): Check that all bits are set.
* src/os.h: Add bits for checking the state of stdin/stdout/stderr.
Add prototypes for OS-specific output sync methods.
* src/posixos.c (check_io_state): Determine the status of stdin,
stdout, stderr an return a suite of bits describing them.
(osync_enabled): If the global variable holding the FD of the lock
file (osync_handle) is valid return true.
(osync_setup): Create a temporary file and remember its name in a
global variable (osync_tmpfile), and set osync_handle.
(osync_get_mutex): If output sync is enabled, return the filename
of the lock file prefixed with "fnm:" to denote a filename.
(osync_parse_mutex): If the provided filename has the wrong format
disable output sync.  Else open the lock file and set osync_handle.
(osync_clear): Close osync_handle.  If we're the parent make, then
also unlink the temporary file.
(osync_acquire): Take a lock on the osync_handle descriptor.
(osync_release): Release the lock on the osync_handle descriptor.
(fd_set_append): Add APPEND mode to a file descriptor.
* src/w32/w32os.c: Perform the same actions as posixos.c, copying
the details from src/w32/compat/posixfcn.c.  Use a mutex rather
than locking a temporary file.
* src/output.h: Remove all the OS-specific content.
* src/output.c: Remove all the OS-specific content.
(set_append_mode): Remove and replace with fd_set_append().
(sync_init): Remove and replace with check_io_state().
(acquire_semaphore): Remove and replace with osync_acquire().
(release_semaphore): Remove and replace with osync_release().
(setup_tmpfile): If the IO state is not obtained, get it.  If stdout
and/or stderr are valid, set up a tempfile to capture them.
(output_init): Set io_state if not set already, and check it when
deciding whether to close stdout on exit.
* src/main.c (main): If we're syncing, set up the mutex using the
new osync_setup() / osync_parse_mutex() methods.
(prepare_mutex_handl_string): Replace with osync_parse_mutex().
(die): Call osync_clear().
* src/w32/compat/posixfcn.c: Remove implementations of fcntl(),
record_sync_mutex(), create_mutex(), and same_stream().
2022-08-30 15:44:43 -04:00
Paul Smith
a2ba5ccbda Add get_tmpfd() and allow anonymous temp files
The output sync feature wants a file descriptor not a FILE*.  We were
using tmpfile() but this returns FILE* which means we needed to dup()
the descriptor then fclose() the original, which is just unnecessary
overhead for every command we run.

Create a get_tmpfd() method that returns a file descriptor directly
by using mkstemp() if available, else do the best we can.

Also allow anonymous temp files if the filename pointer is NULL.
This causes the file to be unlinked.  On Windows this requires a
special open so add an os_anontmp() method to handle this.

* src/makeint.h: Add prototype for get_tmpfd().
* src/misc.c (get_tmpfd): If we have mkstemp() use that, else just
open(2).  If we don't want to keep the filename, unlink the file.
(get_tmpfile): Use get_tmpfd() if we have fdopen(), else use fopen().
* src/output.c (output_tmpfd): Call get_tmpfd() with NULL.
* src/os.h (os_anontmp): On Windows make this a function, else fails.
* src/w32/compat/posixcfn.c (tmpfile): Move to w32os.c:os_anontmp().
* src/w32/w32os.c (os_anontmp): Create a temp file that will be deleted
when the process exits, and return a file descriptor to it.
2022-08-30 15:39:02 -04:00
Dmitry Goncharov
10e130b207 [SV 62908] Don't free fifo_name twice during error handling
* src/posixos.c (jobserver_setup): Set fifo_name to NULL after free.
* tests/scripts/features/jobserver: Add tests.
2022-08-22 21:27:08 -04:00
Paul Smith
7ad2593b2d Support implementing the jobserver using named pipes
Using anonymous pipes for jobserver support has some advantages:
for example there is nothing on disk that needs to be cleaned up.
However it has many obscure problems, related to the fact that in
order for it to work we need to ensure these resources are properly
passed through to child processes that want to use the jobserver.
At the same time we don't want to pass the pipe to process which
DON'T know about the jobserver.

Other processes can open file descriptors which we then think are
our jobserver, but aren't.  And, we open the pipe file descriptors
in blocking mode which doesn't work for all users.

See issues such as SV 57178, SV 57242, and SV 62397

To avoid these issues, use named pipes (on systems where they are
available) instead of anonoymous pipes.  This simplifies many things:
we never need to pass open file descriptors to our children; they
can open the jobserver named pipe.  We don't need to worry about
recursive vs. non-recursive children.  Users don't have to "pass
through" the resources if they are invoking sub-makes.  Each child
can open its own file descriptor and set blocking as needed.

The downside is the named pipe exists on disk and so must be cleaned
up when the "top-level" make instance exits.

In order to allow make to continue to be used in build systems where
older versions of GNU make, or other tools that want to use the
jobserver, but don't understand named pipes, introduce a new option
--jobserver-style that allows the user to choose anonymous pipes.

* NEWS: Announce the change and the --jobserver-style option.
* doc/make.1: Add --jobserver-style documentation.
* doc/make.texi (Special Variables): Add missing items to .FEATURES.
(Options Summary): Add --jobserver-style.
(POSIX Jobserver): Named pipes, changes to --jobserver-auth, and the
--jobserver-style option.
(Windows Jobserver): Document --jobserver-style for Windows.
* configure.ac: Check for mkfifo.
* src/config.h-vms.template: Undefined HAVE_MKFIFO.
* src/config.h.W32.template: Ditto.
* src/main.c: Add jobserver-style as a new command line option.
(main): Add jobserver-fifo to .FEATURES if supported.  Pass the style
option to jobserver_setup().
* src/os.h (jobserver_setup): Accept a style string option.
* src/posixos.c (enum js_type): Enumeration of the jobserver style.
(js_type): Which style we are currently using.
(fifo_name): The path to the named pipe (if in use).
(jobserver_setup): If no style is given, or "fifo" is given, set up a
named pipe: get a temporary file and use mkfifo() on it, then open it
for reading and writing.  If something fails fall back to anonymous
pipes.
(jobserver_parse_auth): Parse jobserver-auth to determine the style.
If we are using a named pipe, open it.  If we're using anonymous pipes
ensure they're valid as before.
(jobserver_get_invalid_auth): Don't invalidate the jobserver when
using named pipes.
(jobserver_clear): Clean up memory used for named pipes.
(jobserver_acquire_all): Unlink the named pipe when done.
* src/w32/w32os.c (jobserver_setup): Check the style argument.
* tests/scripts/features/jobserver: Use --jobserver-style to test
the anonymous pipe behavior, and also test named pipe/semaphore
behavior.  Check invalid jobserver-style options.
* tests/scripts/functions/shell: Use --jobserver-style to test the
anonymous pipe behavior, and also test named pipe/semaphore
behavior.
2022-08-02 23:36:35 -04:00
Paul Smith
09cce75c30 Enhance get_tmpfile() and add get_tmppath()
Move all the logic on creating temporary files into misc.c, and add
a new function get_tmppath() that returns the pathname to a temporary
file without creating or opening it.

* src/makeint.h: Add a declaration for get_tmppath().  Remove the
template argument from get_tmpfile(): it will compute its own.
* src/main.c (main): Remove the logic for computing templates.
* src/vmsjobs.c (child_execute_job): Ditto.
* src/misc.c (get_tmptemplate): New function to return an allocated
template string for use with various mktemp-style functions.
(get_tmppath): Return an allocated path to a temporary file, but do
not create it.  Generally this should be avoided due to TOCTOU issues.
(get_tmpfile): Use get_tmptemplate() to generate a template rather
than using one passed in.  If we don't have mkstemp() then use
get_tmppath() to compute the path of a temp file.
2022-08-02 23:36:35 -04:00
Dmitry Goncharov
07eea3aa49 [SV 62706] Only second-expand targets that might be built
Second-expand only the prerequisites of the targets being built.
Defer second-expanding the prerequisites of targets until we need
to decide if they should be built.

* NEWS: Mention the change in behavior.
* doc/make.texi (Secondary Expansion): Document the new behavior.
* src/filedef.h (struct file): Add flag snapped.
(expand_deps): Declare a function to second expand the
prerequisites of a target.
* src/file.c (rehash_file): Merge flag snapped.
(expand_deps): Remove qualifier static. Check flag snapped.
(snap_deps): Remove the loop which performed second expansion for all
targets.
* src/remake.c (update_file_1): Second expand the prerequisites of
the considered target.
* tests/scripts/features/se_explicit: Add tests.
* tests/scripts/features/se_implicit: Ditto.
* tests/scripts/features/se_statpat: Ditto.
2022-07-30 18:40:28 -04:00
Paul Smith
16e14b4114 Disable the jobserver in non-recursive children
Savannah issues such as SV 57242 and SV 62397 show how passing
references to closed file descriptors via the --jobserver-auth option
in MAKEFLAGS can lead to problematic outcomes.

When computing the child environment for a non-recursive shell, add
an extra option to MAKEFLAGS to disable the file descriptors for the
jobserver.

Unfortunately this doesn't modify the value of the make variable
MAKEFLAGS, it only modifies the value of the sub-shell environment
variable MAKEFLAGS.  This can lead to confusion if the user is not
considering the distinction.

* src/makeint.h: Publish the jobserver-auth value.  Add a global
definition of the name of the command line option.
* src/os.h (jobserver_get_invalid_auth): New function to return a
string invalidating the jobserver-auth option.
* src/w32/w32os.c (jobserver_get_invaid_auth): Implement it.  On
Windows we use a semaphore so there's no need to invalidate.
* src/posixos.c (jobserver_parse_auth): If we parse the invalid
auth string, don't set up the jobserver.
(jobserver_get_invalid_auth): Return an invalid option.
* src/variable.h (target_environment): Specify if the target
environment is for a recursive shell or non-recursive shell.
* src/variable.c (target_environment): Move checking for MAKELEVEL
into the loop rather than doing it at the end.
Along with this, check for MAKEFLAGS and MFLAGS, and update them
based on whether we're invoking a recursive or non-recursive child,
and also on whether it's necessary to invalidate the jobserver.
* src/function.c (func_shell_base): Shell functions can never be
recursive to pass 0 to target_environment().
* src/job.c (start_job_command): Specify whether the child is
recursive when calling target_environment().
* src/main.c: Export jobserver_auth.  sync_mutex doesn't need to
be exported.  Use the global definition for the option name.
* tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
2022-07-30 18:40:28 -04:00
Paul Smith
77881d2281 Ensure that MAKEFLAGS is set when invoking $(shell ...)
* src/main.c (main): Don't reset the jobserver if the number of
slots has not changed.
(define_makeflags): Add all normal flags even when ALL is not set.
* tests/scripts/functions/shell: Test invoking make in $(shell ...).
* tests/scripts/variables/MAKEFLAGS: Test the value of MAKEFLAGS in
$(shell ...).
2022-07-30 18:40:28 -04:00
Paul Smith
2fe96e4a41 Take advantage of mempcpy() and stpcpy()
* src/makeint.h (stpcpy): Add missing declaration.
* src/amiga.c (MyExecute): Avoid extra strlen using stpcpy.
* src/function.c (func_shell_base): Ditto.
(func_error): Use memcpy() not strcpy() when we know the length.
* src/job.c (construct_command_argv_internal): Use stpcpy().
* src/main.c (main): Ditto.
(define_makeflags): Ditto.
* src/variable.c (print_target_variables): Use memcpy() when we know
the length.
* src/commands.c (set_file_variables): Use mempcpy().
* src/expand.c (variable_buffer_output): Ditto.
* src/file.c (expand_deps): Ditto.
* src/function.c (abspath): Ditto.
(handle_function): Ditto.
* src/implicit.c (pattern_search): Ditto.
* src/job.c (construct_command_argv_internal): Use mempcpy() and
don't add multiple spaces when there are no shell flags.
* src/main.c (decode_env_switches): Use mempcpy() to simplify.
(define_makeflags): Ditto.
* src/variable.c (selective_vpath_search): Ditto.
2022-07-30 18:40:21 -04:00
Paul Smith
dd24a4c1cf Convert HAVE_GETLOADAVG to HAVE_DECL_GETLOADAVG
We want to know if getloadavg() is declared, not if we have it.

* configure.ac: Add it as a define (why is this not the default?)
* src/job.c: Conditionalize declaration on HAVE_DECL_GETLOADAVG.
* config.ami.template: Rename HAVE_GETLOADAVG to HAVE_DECL_GETLOADAVG.
* config.h-vms.template: Ditto.
* config.h.W32.template: Ditto.
Clean up and add new elements from latest config.h.
2022-07-09 11:09:17 -04:00
Paul Smith
5dc7358547 * configure.ac: Remove AC_FUNC_SETVBUF_REVERSED.
This macro is obsolete: no useful system has this problem anymore.
* src/output.c (output_init): Remove reference to SETVBUF_REVERSED.
* src/config.ami.template: Remove undef of SETVBUF_REVERSED.
* src/config.h-vms.template: Ditto.
* src/config.h.W32.template: Ditto.
2022-07-09 10:47:13 -04:00
Paul Smith
e33af0fb4a * configure.ac: Check for stpcpy() support.
* src/misc.c (stpcpy): Define it if not provided.
2022-07-09 10:47:13 -04:00
Paul Smith
6f7e06ec4e getloadavg: Remove support for privileged invocation
This was needed when getloadavg required privileged access; in this
case GNU make needed to be installed as a setgid program.  But this
hasn't been supported by gnulib getloadavg() since 2011 and systems
are no longer using it, so remove it.

* src/makeint.h (user_access): Remove unnecessary function.
(make_access): Ditto.
(child_access): Ditto.
* src/misc.c: Remove implementations of the *_access() functions.
* src/main.c (main): Remove unneeded call to user_access().
* src/job.c (load_too_high): Remove calls to {make,user}_access().
(exec_command): Remove call to child_access().
* src/remote-cstms.c: Remove calls to these methods.  I suppose it
might be possible this is needed and was piggy-backing on the
privileged setting but since that's been broken for a while I doubt
this is needed.  If so we can bring back the implementation into
this source file.
* src/config.h.W32.template: Remove GETLOADAVG_PRIVILEGED undef.
* src/config.h-vms.template: Ditto.
* src/config.ami.template: Ditto.
2022-07-09 10:47:13 -04:00
Paul Smith
9992cb0b83 bootstrap: Remove strerror()
* bootstrap.conf: Remove strerror module
* configure.ac: Add a check for strerror
* src/misc.c: Add a default strerror() if not found
2022-07-09 10:46:47 -04:00
Paul Smith
0cbee1b475 bootstrap: Remove strtoll()
This pulls in entirely too much stuff we don't need.  Instead grab
just the gnulib source file, then include it in src/misc.c.

* bootstrap.conf: Add just the lib/strtol.c source file.
* configure.ac: Check for strtoll.
* src/misc.c: Include strtol.c, with QUAD set, if needed.
2022-07-09 10:46:47 -04:00
Paul Smith
0793658c09 Run autoupdate and clean up autoconf usage
We can assume that the return type of a signal handler is void.
We can assume that, if sys/time.h exists, it can be included
with time.h.

* bootstrap: Get the latest version
* configure.ac: Require a newer version of autoconf.
Remove unnecessary AC_PROG_CC_C99 (already have AC_PROC_CC).
Remove unnecessary AC_AIX, AC_ISC_POSIX, AC_MINIX.
Remove unnecessary AC_HEADER_STDC, AC_HEADER_TIME, AC_TYPE_SIGNAL.
Use strerror to search for the cposix library.
* src/commands.c (fatal_error_signal): Assume return type is void.
* src/commands.h: Ditto.
* src/job.c: Ditto.
* src/job.h: Ditto.
* src/main.c: Ditto.
* src/makeint.h: Ditto.
Don't bother with TIME_WITH_SYS_TIME.
* src/remote-cstms.c: Check HAVE_SYS_TIME_H.
* src/config.ami.template: Remove RETSIGTYPE.
* src/config.h-vms.template: Ditto.
* src/config.h.W32.template: Ditto.
Remove TIME_WITH_SYS_TIME.
2022-07-09 10:46:47 -04:00
Paul Smith
98da874c43 [SV 10593] Export variables to $(shell ...) commands
Export all variables, including exported makefile variables, when
invoking a shell for the $(shell ...) function.  If we detect a
recursive variable expansion, silently ignore that variable and do
not export it.  We do print a debug message.

* NEWS: Announce the potential backward-incompatibility.
* doc/make.texi (Shell Function): Document the export behavior.
* src/main.c (main): Add "shell-export" to .FEATURES.
* src/job.h: New function to free struct childbase.
* src/job.c (free_childbase): Implement it; call from free_child.
* src/function.c (func_shell_base): Use target_environment() to
obtain the proper environment for the shell function.
Use free_childbase() to free memory.
(windows32_openpipe): Don't reset the environment: the caller
already provided a proper PATH variable in envp.
* src/variable.c (target_environment): If we detect a recursive
expansion and we're called from func_shell, ignore the variable.
(sync_Path_environment): Simplify and reduce memory allocation.
* tests/scripts/functions/shell: Add tests for this.
2022-07-09 10:44:21 -04:00
Dmitry Goncharov
88d6c22a48 [SV 62650] Show correct warning when combining targets
* src/file.c (rehash_file): Fix warning message.
(rehash_file): Fix comment to match the behavior.
* tests/scripts/features/se_explicit: Fix test.
* tests/scripts/features/mult_rules: Add a new test.
2022-07-09 10:44:00 -04:00
Dmitry Goncharov
c3b39d0654 [SV 62514] Honor command line interface flags
Commit f2771aa614 introduced a bug where some switches were left out
of MAKEFLAGS.  Instead of resetting switches, get the same results by
filtering out duplicates.

* src/makeint.h: Remove reset_switches.
* src/main.c: (reset_switches): Remove reset_switches.
* (main): Remove call to reset_switches.
* (decode_switches): Filter out duplicate flags.
* src/variable.c: (set_special_var):  Remove call to reset_switches.
* tests/scripts/variables/MAKEFLAGS: Verify that duplicate flags are
properly filtered out.
2022-07-09 10:43:33 -04:00
Paul Smith
3ec497f8f8 Don't add GNUMAKEFLAGS to the environment
If GNUMAKEFLAGS was not present in the environment when we started,
don't add it.

* src/main.c (main): Don't mess with GNUMAKEFLAGS unless it exists.
* tests/scripts/variables/GNUMAKEFLAGS: Test this behavior.
2022-06-19 14:35:27 -04:00
Paul Smith
97e51c0285 Avoid overwriting buffers with long pathnames
Reported, with initial patch, by Gisle Vanem <gvanem@online.no>

* src/main.c (find_and_set_default_shell) [W32]: Pass search_token
directly to w32ify: no need to make a copy first.  When we need to
construct a path, use snprintf() to be sure we don't overwrite
the locally-allocated buffer.
* src/w32/pathstuff.c (w32ify) [W32]: Use the malloc version of
_fullpath(), followed by strncpy(), to avoid overwriting buffers.
2022-06-17 19:55:11 -04:00
Sergei Trofimovich
621d3196fa [SV 62100] Add '--shuffle' option support
Introduce non-deterministic ordering into goal and prerequisite
traversal to help tease out inconsistent failures that may happen
when running in parallel build mode.

Introduce second order into each dependency chain:
1. Existing order is syntactic order reachable via 'dep->next'
2. New order is shuffled order stored as 'dep->shuf' in each 'dep'

When updating goals and prerequisites and '--shuffle' is provided,
use the shuffled order to walk the graph.  When automatic variable
are set always use the syntactic order of parameters.

* Makefile.am: Add new src/shuffle.c and src/shuffle.h file.
* build_w32.bat: Ditto.
* builddos.bat: Ditto.
* makefile.com: Ditto.
* po/POTFILES.in: Ditto.
* doc/make.texi: Add documentation for --shuffle.
* doc/make.1: Ditto.
* src/dep.h (DEP): Add the shuf pointer.
* src/filedef.h (struct file): Add was_shuffled flag.
* src/main.c: (shuffle_mode): Global flag for the shuffle mode.
(usage): Add the --shuffle option.
(switches): Ditto.
(main): Set shuffle_mode based on the command line parameter.
Reshuffle prerequisites if requested.
* src/remake.c (update_goal_chain): Walk the shuffled list if enabled.
(update_file_1): Ditto.
* src/shuffle.h: Provide an interface for shuffling prerequisites.
* src/shuffle.c: Implement option parsing and prerequisite shuffling.
* tests/scripts/options/shuffle: Test shuffle option and modes.
2022-06-04 19:04:37 -04:00
Paul Smith
e4b7ac21dc * src/misc.c (make_toui): Parse a string into an unsigned int
* src/makeint.h: Declare it.
* src/arscan.c (ar_scan): Replace atoi() calls with make_toui().
Modify some integral types to be more correct.
* src/job.c (load_too_high): Replace atoi() calls with make_toui().
* src/main.c (main): Ditto.
(decode_switches): Ditto.
2022-06-04 18:34:15 -04:00
Paul Smith
d444b87173 * src/misc.c (make_pid): A function to return the PID
* src/makeint.h: Declare it.
* src/commands.c (fatal_error_signal): Call it.
2022-06-04 17:04:38 -04:00
Noah Goldstein
5690084634 Replace strcmp() with memcmp() where possible
memcmp() is always faster than strcmp().  In places where we already
know that both buffers have sufficient size, replace strcmp() with
memcmp().

* src/main.c (main): Replace strncmp() with memcmp()==0.
* src/read.c (word1eq): Ditto.
* src/commands.c (set_file_variables): Ditto.
* src/function.c (func_filter_filterout): Ditto.
(a_word_hash_cmp): Use STRING_N_COMPARE since we know the length.
(func_sort): Replace strcmp() with memcmp().
2022-04-24 17:52:54 -04:00
Noah Goldstein
9fa63eb918 hash: Remove unnecessary isupper() check before tolower()
The standard requires that tolower() returns its argument if there's
no valid lowercase conversion: it's not necessary to check isupper()
before invoking tolower().

Comparing the performance of the old ISTRING_HASH to the new one
on all the files present in GLIBC.

  Data Type            Old Time   New Time
  Alternating case    24985.754  13883.422
  random case         35211.777  13569.051
  all lower           18818.974  13706.645
  all upper           38859.454  13506.315

* src/hash.h (ISTRING_HASH_1): Omit isupper() check.
(ISTRING_HASH_2): Ditto.
2022-04-24 17:52:54 -04:00
Dmitry Goncharov
668eda0527 [SV 62206] Fix %-substitution in second expansion of pattern rules
During second expansion of pattern rules only the first pattern in
each "group" was being substituted.  E.g. in this makefile:

  .SECONDEXPANSION:
  all: hello.x
  %.x: $$(wordlist 1, 99, %.1 %.%.2) ; $(info $@ from $^)
  hello.1 hello.\%.2 \%.1 \%.\%.2: ;

the output would build "hello.1" and "%.%.2" because each function
is considered a single "word" and only the first pattern is replaced.

Fix the expansion so each whitespace-separated string is considered a
word and the first pattern is replaced, giving "hello.1" and
"hello.%.2".

* src/rule.c (snap_implicit_rules): Keep enough space to replace %
with $(*F) if necessary.
* src/implicit.c (pattern_search): During second expansion break each
get_next_word result into individual words and replace the first % in
each with $* or $(*F) as needed.
* tests/scripts/features/patternrules: Add tests for variations.
2022-04-24 17:52:07 -04:00
Dmitry Goncharov
f14824878d [SV 62356] If -R is set in the makefile, disable -r
If -R is given on the command line it implies -r as well; make this
same assumption if -R is provided in MAKEFLAGS set in the makefile.

* src/main.c (main): Check no_builtin_variables_flag after reading
makefiles.
* tests/scripts/options/dash-r: Add a test.
2022-04-24 10:39:32 -04:00
Dmitry Goncharov
8c2aa889bb [SV 62324] Simplify set_file_variables by passing in the stem
Previously we always used the file->stem value as our stem in
set_file_variables(); when that wasn't correct we had to temporarily
set that value while the function was called, then reset it afterward.
This led to issues (for example when we assumed the stem was a cached
string but it wasn't).

Avoid this by passing in the stem as an argument so that different
values can be provided.

Add tests to verify this.

* src/commands.c (set_file_variables): Take second parameter stem to
relieve the callers of set_file_variables() from setting/restoring
file->stem.
* src/commands.h (set_file_variables): Ditto.
(execute_file_commands): Pass file->stem to set_file_variables().
* src/file.c (expand_deps): Pass d->stem to set_file_variables() and
remove set and restore of file->stem.
* src/implicit.c (pattern_search): Pass stem to set_file_variables()
and remove set and restore of file->stem.
* tests/scripts/features/se_explicit: Add new tests.
* tests/scripts/features/se_implicit: Ditto.
* tests/scripts/features/se_statpat: Ditto.
* tests/scripts/variables/automatic: Ditto.
2022-04-24 10:39:32 -04:00
Dmitry Goncharov
bb21dd4d2d [SV 62278] Only expand the first pattern during secondary expansion
During normal pattern rule expansion only the first pattern (%) is
expanded; however during secondary expansion all patterns were
expanded.  Modify secondary expansion to match the behavior of normal
expansion.

Implementation tweaked by Paul Smith <psmith@gnu.org>

* src/file.c (expand_deps): Don't use subst_expand() which replaces
all % with $*: instead replace only the first one, by hand.
Fix a memory leak where the dep structure was not always freed.
* tests/scripts/features/statipattrules: Use .RECIPEPREFIX not TAB.
Add a series of tests verifying that static pattern rules with and
without secondary expansion both return the same results.
2022-04-24 10:39:32 -04:00
Paul Smith
93fd0c0c9a * src/commands.h: [SV 62213] Forward-declare struct file / child 2022-04-24 10:39:32 -04:00
Dmitry Goncharov
634ef057ad * src/read.c (check_specials): [SV 62171] Reset ARFLAGS under .POSIX 2022-04-24 10:39:32 -04:00
Dmitry Goncharov
949c0464a9 [SV 62145] Remove a stdin temp file on re-exec failure.
If the re-exec fails, be sure to remove a temp makefile that was
created to read from stdin.

* src/job.c (exec_command): Return on failure.
(child_execute_job): Call exit if exec_command returns.
* src/job.h (exec_command): Don't mark as NORETURN.
* src/main.c (main): Unlink stdin temporary file if re-exec fails.
* tests/run_make_tests.pl: Get value for ERR_nonexe_file/ERR_exe_dir.
* tests/scripts/features/temp_stdin: Test that temp file unlink works.
2022-04-24 10:39:32 -04:00
Paul Smith
667d70eac2 [SV 62118] Correctly handle -f- options on re-exec
The -f, -file, and --makefile options were not properly handled when
re-exec'ing due to makefile updates.  This problem, plus a patch and
tests, was reported by Dmitry Goncharov <dgoncharov@users.sf.net>.

While examining this I found another bug: after re-exec we forgot the
batch file was temporary and never deleted it.

I decided to fix all these problems at once using a different fix
than Dmitry's: I created a new internal-only command-line option,
--temp-stdin.  When reconstructing the make options for a re-exec,
replace the -f/--file/--makefile option that reads from stdin with
--temp-stdin=<filename> so that the re-exec'd version of make knows
it's a temporary batch file and will delete it.

We no longer need to add the -o options because the re-exec'd make
knows this is a temporary makefile and treats it as such.

To simplify, replace the --file and --makefile options taking a
filename, with just -f<filename> on re-exec.

Some examples of the rewrite:

  User command line               Re-exec command line
  -----------------               --------------------
  -f-                             --temp-stdin=<batch>
  --file -                        --temp-stdin=<batch>
  -f - --makefile a.mk            --temp-stdin=<batch> -fa.mk
  --file=a.mk                     -fa.mk
  -fa.mk                          -fa.mk
  -Rf a.mk                        -Rf a.mk
  -Rf-                            -R --temp-stdin=<batch>

* src/main.c (stdin_offset): Remember the offset into the makefiles
list of the batch file read from stdin.  Remove stdin_nm.
(struct command_switch): Create a new --temp-stdin option, which
also updates the makefiles list.
(main): Add the temporary filename to the string cache.
Move the tempfile handling after checking makefile arguments for "-"
so that files provided via --temp-stdin are also handled specially.
When rewriting re-exec options, we may need one more than we had
originally so create a new argv list.  Walk through the original
list and convert it to the new list, following the above process.
(decode_switches): Set the stdin_offset flag if we see --temp-stdin.
* tests/scripts/options/dash-f: Add many more tests, provided by
Dmitry Goncharov <dgoncharov@users.sf.net>.
2022-02-27 18:01:13 -05:00
Paul Smith
236589642e Remove extraneous characters from fatal() calls
The fatal() method adds ".  Stop.\n" to every message.

* src/amiga.c (MyExecute): Remove newline from fatal() message.
* src/job.c (<various>): Ditto.
* src/file.c (snap_deps): Remove "." from fatal() message.
* src/main.c (main): Ditto.
* src/load.c: Ditto.
* tests/scripts/targets/NOTINTERMEDIATE: Fix expected output.
2022-02-27 18:01:13 -05:00
Paul Smith
cb9ac2b0cc * src/main.c: Ensure the stdin temp file is deleted when dying. 2022-02-27 18:01:13 -05:00
Dmitry Goncharov
40a6751abf [SV 62088] Avoid re-exec due to stdin.
The fix for SV 60595 causes make to re-execute when the makefile is
read from stdin.  E.g.
    $ printf 'all:; $(info hello)' | make -sf -

Reported by Sergei Trofimovich <siarheit@google.com>
Test added by Paul Smith <psmith@gnu.org>

* src/main.c (main): Set the mtime of the stdin temporary file.
* tests/scripts/features/reinvoke: Add a test.
2022-02-27 18:00:47 -05:00
Paul Smith
8cdaabbf8e tests: Preserve Windows temp environment variables
* src/main.c (main): Show the temp filename on error.  Also on
Windows prefer TMP, then TEMP, and only lastly TMPDIR.
* tests/test_driver.pl: Add TMP, TEMP, and USERPROFILE to the list
of environment variables to preserve.
2022-02-27 17:47:19 -05:00
Paul Smith
8b3e678ace Ensure buffers are large enough for integer values
Issue raised by Sergei Trofimovich <siarheit@google.com>

* src/makeint.h (INTSTR_LENGTH): Update for unsigned values.
* src/function.c (func_lastword): Use INTSTR_LENGTH as buffer size.
(shell_function_completed): Ditto.
(func_call): Ditto.
* src/remote-cstms.c (start_remote_job): Ditto.
2022-02-21 09:29:41 -05:00
Paul Smith
ec09ec775a * <all>: Update copyright notices. 2022-02-10 14:48:26 -05:00
Paul Smith
342a9bb54b Don't write $(shell ...) stdout to stderr on failure
If a $(shell ...) invocation failed due to a command-not-found error,
make wrote the stdout of that shell to our stderr for some reason.
That seems very wrong.

If the command's stderr was not redirected then its output would have
already been written to its stderr, and if it was redirected then we
shouldn't take it upon ourselves to force it to go to stderr!

* src/function.c (func_shell_base): Append shell stdout even if the
shell command failed.
* tests/run_make_tests.pl: Determine the error generated for
command-not-found situations.
* tests/scripts/functions/shell: Verify that redirecting stderr to
stdout will behave properly if the command is not found.
2022-02-06 18:46:32 -05:00
Dmitry Goncharov
1fd0bfaf1e [SV 61805] Support quoted whitespace in .SHELLFLAGS with .ONESHELL
* src/job.c (construct_command_argv_internal): Call recursively to
parse .SHELLFLAGS when .ONESHELL is active.
* tests/scripts/targets/ONESHELL: Add tests.
2022-02-06 18:46:32 -05:00
Dmitry Goncharov
d34f46c1f3 [SV 48643] Make prereqs mentioned in unrelated rules explicit
* src/implicit.c (pattern_search): Set is_explicit when a prereq
is mentioned in an unrelated rule.
2022-02-06 18:46:32 -05:00
Dmitry Goncharov
f0ab709640 * src/implicit.c (pattern_search): [SV 61864] Fix debug output. 2022-02-06 18:46:32 -05:00
Paul Smith
5f2d114c6d Avoid interleaved $(info ) output
Since $(info ) function output is written using two system calls for
the message and the newline, it's possible for output from another
parallel make job to sneak in between them.

Reported by Paul Eggert <eggert@cs.ucla.edu>, who saw the report from
Lars Ingebrigtsen <https://bugs.gnu.org/53358>.

* src/function.c (func_error): Paul provided a fix but instead I
rewrote the entire function: it's not possible for it to be invoked
with anything other than exactly one argument so don't worry about
re-combining the arguments.
2022-02-06 18:46:32 -05:00
Paul Smith
f485881d4a * src/implicit.c (pattern_search): Use const ptrs when possible.
Change suggested by Joe Filion <joefilion4@gmail.com>
2022-02-06 18:46:32 -05:00
Paul Smith
15db387f18 * src/job.c (load_too_high): Re-enable Linux /proc/loadavg checks.
Further experimentation shows that my previous thinking that there
was a problem using this was woefully mistaken.  The value generated
by the kernel shows runn*ABLE* processes not runn*ING* processes.
* NEWS: Announce the change.
2022-02-06 18:46:32 -05:00
Jouke Witteveen
2b25eac587 * src/function.c (parse_textint): Handle ints without 0 properly.
* tests/scripts/functions/intcmp: Add tests for values without 0.
2022-01-17 19:11:59 -05:00
Paul Smith
e72c5e021f [SV 61226] Revert changes to detect missing included files
The fix for SV 60595 introduced a backward-incompatibility: rules that
purported to rebuild included files, but didn't actually do so, were
treated as errors whereas before they were ignored.  This breaks a
common idiom in makefiles where an empty recipe is created for an
included makefile so make doesn't complain if it doesn't exist.

Unfortunately this means make cannot diagnose some types of errors.

Extra tests supplied by Dmitry Goncharov <dgoncharov@users.sf.net>.

* doc/make.texi (Including Other Makefiles): Clarify this behavior.
* src/main.c (main): Don't run the new check-for-errors behavior.
* tests/scripts/features/reinvoke: Reset tests of the "old" behavior
and add new tests for this situation.
2022-01-17 18:45:47 -05:00