In this mode we still collect all the output from a given target and
dump it at once. However we don't treat recursive lines any differently
from non-recursive lines. Also we don't print enter/leave messages
after every dump. However we do ensure that we always print them once
to stdout, so the parent make will collect it properly.
Create a new file, output.c, and collect functions that generate output there.
We introduce a new global context specifying where output should go (to stdout
or to a sync file), and the lowest level output generator chooses where to
write output based on that context.
This allows us to set the context globally, and all operations that write
output (including functions like $(info ...) etc.) will use it.
Removed the "--trace=dir" capability. It was too confusing. If you have
directory tracking enabled then output sync will print the enter/leave message
for each synchronized block. If you don't want that, disable directory
tracking.
In various places we were passing flags and characters to compare, then
using complex conditionals to see where to stop in string searches.
Performance numbers reveal that we were spending as much as 23% of our
processing time in these functions, most of it in the comparison lines.
Instead create a character map and use a single bitwise comparison to
determine if this is any one of the stop characters.
This mode replaces the previous heuristic setting enabled with -O, where we
would log directory enter/leave for each synchronized output. Now we only
do that if --trace=dir is given.
job.c (child_out): Output the newline following the message
before fllush-ing the stream. Avoids displaying the following
failure message, which goes to stderr, on the same line.
GCC was giving us warnings, most OS's now just run fork() when you call
vfork(), and looking at the standard definition of vfork() we are a long way
from using it safely anyway: you're not allowed to even call a function before
you exec().
If output-sync is enabled, have make write the command line to the temp file
instead of printing it directly to the screen to ensure that the output is
ordered properly. Also, remove extraneous enter/leave operations by having
them printed directly when dumping temp file output.
If we are not going to sync a command line then dump any collected output
first to preserve ordering. Do some code cleanup:
* Move the handle init to a separate function.
* Move the temp file truncation to the output function.
* Remember whether we sync in a variable for readability.
* Handle EINTR and short writes in child_out().
* Always call sync_output() in case output_sync was changed due to error.
job.c (start_job_command): Make the condition for creating a
temporary output file be identical to the Posix code branch.
Suggested by Frank Heckenbach <f.heckenbach@fh-soft.de>.
job.c (construct_command_argv_internal) [WINDOWS32]: Return
right after generating new_argv for one_shell case. This fixes
the Windows build for both Unixy shell and stock Windows shells.
Enhance the child_error() function so that it will write error output to the
child's sync output buffer, if it exists. If it doesn't the output goes to
stdout/stderr.
A new flag to the -O/--output-sync, "job", selects a per-job (that is, per
line of a recipe) output synchronization. To support this move the close of
the temp file out of the sync_output() function and don't do it until we free
the child, since we may call sync_output() multiple times in a given recipe.
When we set up for a new temp file, if we're in per-job mode we truncate the
file and seek to the beginning to re-use it for every job.
read.c (record_files): Pay attention to .ONESHELL in MS-Windows.
job.c (construct_command_argv_internal): Support .ONESHELL on
MS-Windows, when the shell is not a Unixy shell.
w32/compat/posixfcn.c: New file, with emulations of Posix
functions and Posix functionality for MS-Windows.
w32/subproc/sub_proc.c: Include io.h.
(process_noinherit): New function, forces a file descriptor to not
be inherited by child processes.
(process_easy): Accept two additional arguments, and use them to
set up the standard output and standard error handles of the child
process.
w32/include/sub_proc.h (process_easy): Adjust prototype.
(process_noinherit): Add prototype.
read.c [WINDOWS32]: Include windows.h and sub_proc.h.
makeint.h (LOCALEDIR) [WINDOWS32}: Define to NULL if not
defined. This is needed because the MS-Windows build doesn't have
a canonical place for LOCALEDIR.
(WIN32_LEAN_AND_MEAN) [WINDOWS32]: Define, to avoid getting from
windows.h header too much stuff that could conflict with the code.
main.c <sync_mutex>: New static variable.
<switches>: Add support for "--sync-mutex" switch.
(decode_output_sync_flags): Decode the --sync-mutex= switch.
(prepare_mutex_handle_string) [WINDOWS32]: New function.
(main): Add "output-sync" to .FEATURES.
job.h (CLOSE_ON_EXEC) [WINDOWS32]: Define to call
process_noinherit.
(F_GETFD, F_SETLKW, F_WRLCK, F_UNLCK, struct flock) [WINDOWS32]:
New macros.
(RECORD_SYNC_MUTEX): New macro, a no-op for Posix platforms.
(sync_handle_t): New typedef.
job.c <sync_handle>: Change type to sync_handle_t.
(FD_NOT_EMPTY): Seek to the file's end. Suggested by Frank
Heckenbach <f.heckenbach@fh-soft.de>.
(pump_from_tmp_fd) [WINDOWS32]: Switch to_fd to binary mode for
the duration of this function, and then change back before
returning.
(start_job_command) [WINDOWS32]: Support output_sync mode on
MS-Windows. Use a system-wide mutex instead of locking
stdout/stderr. Call process_easy with two additional arguments:
child->outfd and child->errfd.
(exec_command) [WINDOWS32]: Pass two additional arguments, both
-1, to process_easy, to adjust for the changed function signature.
function.c (windows32_openpipe) [WINDOWS32]: This function now
returns an int, which is -1 if it fails and zero otherwise. It
also calls 'error' instead of 'fatal', to avoid exiting
prematurely.
(func_shell_base) [WINDOWS32]: Call perror_with_name if
windows32_openpipe fails, now that it always returns. This avoids
a compiler warning that error_prefix is not used in the MS-Windows
build.
config.h.W32.template (OUTPUT_SYNC): Define.
build_w32.bat: Add w32/compat/posixfcn.c to compilation and
linking commands.
From Frank Heckenbach <f.heckenbach@fh-soft.de>:
job.c (sync_output): Don't discard the output if
acquire_semaphore fails; instead, dump the output unsynchronized.
sh_chars_dos[]. Ignore an escaped backslash inside a string
quoted with "..". This lifts the 4KB or 8KB command-line length
limitation imposed by the Windows shell when a command uses quoted
strings, because we now don't call the shell in that case.
job.c (create_batch_file): Declare the counter of batch files
static, to avoid having 2 jobs using the same file name and
stepping on each other's toes. When all 64K names are used up,
make one more loop looking for slots that became vacant. This
avoids leaving behind temporary batch files in the temporary
directory, which happens frequently on a fast machine when using
parallel builds.
