In this mode we still collect all the output from a given target and
dump it at once. However we don't treat recursive lines any differently
from non-recursive lines. Also we don't print enter/leave messages
after every dump. However we do ensure that we always print them once
to stdout, so the parent make will collect it properly.
Create a new file, output.c, and collect functions that generate output there.
We introduce a new global context specifying where output should go (to stdout
or to a sync file), and the lowest level output generator chooses where to
write output based on that context.
This allows us to set the context globally, and all operations that write
output (including functions like $(info ...) etc.) will use it.
Removed the "--trace=dir" capability. It was too confusing. If you have
directory tracking enabled then output sync will print the enter/leave message
for each synchronized block. If you don't want that, disable directory
tracking.
In various places we were passing flags and characters to compare, then
using complex conditionals to see where to stop in string searches.
Performance numbers reveal that we were spending as much as 23% of our
processing time in these functions, most of it in the comparison lines.
Instead create a character map and use a single bitwise comparison to
determine if this is any one of the stop characters.
This mode replaces the previous heuristic setting enabled with -O, where we
would log directory enter/leave for each synchronized output. Now we only
do that if --trace=dir is given.
job.c (child_out): Output the newline following the message
before fllush-ing the stream. Avoids displaying the following
failure message, which goes to stderr, on the same line.
GCC was giving us warnings, most OS's now just run fork() when you call
vfork(), and looking at the standard definition of vfork() we are a long way
from using it safely anyway: you're not allowed to even call a function before
you exec().
If output-sync is enabled, have make write the command line to the temp file
instead of printing it directly to the screen to ensure that the output is
ordered properly. Also, remove extraneous enter/leave operations by having
them printed directly when dumping temp file output.
If we are not going to sync a command line then dump any collected output
first to preserve ordering. Do some code cleanup:
* Move the handle init to a separate function.
* Move the temp file truncation to the output function.
* Remember whether we sync in a variable for readability.
* Handle EINTR and short writes in child_out().
* Always call sync_output() in case output_sync was changed due to error.
job.c (start_job_command): Make the condition for creating a
temporary output file be identical to the Posix code branch.
Suggested by Frank Heckenbach <f.heckenbach@fh-soft.de>.
job.c (construct_command_argv_internal) [WINDOWS32]: Return
right after generating new_argv for one_shell case. This fixes
the Windows build for both Unixy shell and stock Windows shells.
Enhance the child_error() function so that it will write error output to the
child's sync output buffer, if it exists. If it doesn't the output goes to
stdout/stderr.
A new flag to the -O/--output-sync, "job", selects a per-job (that is, per
line of a recipe) output synchronization. To support this move the close of
the temp file out of the sync_output() function and don't do it until we free
the child, since we may call sync_output() multiple times in a given recipe.
When we set up for a new temp file, if we're in per-job mode we truncate the
file and seek to the beginning to re-use it for every job.
read.c (record_files): Pay attention to .ONESHELL in MS-Windows.
job.c (construct_command_argv_internal): Support .ONESHELL on
MS-Windows, when the shell is not a Unixy shell.
w32/compat/posixfcn.c: New file, with emulations of Posix
functions and Posix functionality for MS-Windows.
w32/subproc/sub_proc.c: Include io.h.
(process_noinherit): New function, forces a file descriptor to not
be inherited by child processes.
(process_easy): Accept two additional arguments, and use them to
set up the standard output and standard error handles of the child
process.
w32/include/sub_proc.h (process_easy): Adjust prototype.
(process_noinherit): Add prototype.
read.c [WINDOWS32]: Include windows.h and sub_proc.h.
makeint.h (LOCALEDIR) [WINDOWS32}: Define to NULL if not
defined. This is needed because the MS-Windows build doesn't have
a canonical place for LOCALEDIR.
(WIN32_LEAN_AND_MEAN) [WINDOWS32]: Define, to avoid getting from
windows.h header too much stuff that could conflict with the code.
main.c <sync_mutex>: New static variable.
<switches>: Add support for "--sync-mutex" switch.
(decode_output_sync_flags): Decode the --sync-mutex= switch.
(prepare_mutex_handle_string) [WINDOWS32]: New function.
(main): Add "output-sync" to .FEATURES.
job.h (CLOSE_ON_EXEC) [WINDOWS32]: Define to call
process_noinherit.
(F_GETFD, F_SETLKW, F_WRLCK, F_UNLCK, struct flock) [WINDOWS32]:
New macros.
(RECORD_SYNC_MUTEX): New macro, a no-op for Posix platforms.
(sync_handle_t): New typedef.
job.c <sync_handle>: Change type to sync_handle_t.
(FD_NOT_EMPTY): Seek to the file's end. Suggested by Frank
Heckenbach <f.heckenbach@fh-soft.de>.
(pump_from_tmp_fd) [WINDOWS32]: Switch to_fd to binary mode for
the duration of this function, and then change back before
returning.
(start_job_command) [WINDOWS32]: Support output_sync mode on
MS-Windows. Use a system-wide mutex instead of locking
stdout/stderr. Call process_easy with two additional arguments:
child->outfd and child->errfd.
(exec_command) [WINDOWS32]: Pass two additional arguments, both
-1, to process_easy, to adjust for the changed function signature.
function.c (windows32_openpipe) [WINDOWS32]: This function now
returns an int, which is -1 if it fails and zero otherwise. It
also calls 'error' instead of 'fatal', to avoid exiting
prematurely.
(func_shell_base) [WINDOWS32]: Call perror_with_name if
windows32_openpipe fails, now that it always returns. This avoids
a compiler warning that error_prefix is not used in the MS-Windows
build.
config.h.W32.template (OUTPUT_SYNC): Define.
build_w32.bat: Add w32/compat/posixfcn.c to compilation and
linking commands.
From Frank Heckenbach <f.heckenbach@fh-soft.de>:
job.c (sync_output): Don't discard the output if
acquire_semaphore fails; instead, dump the output unsynchronized.
sh_chars_dos[]. Ignore an escaped backslash inside a string
quoted with "..". This lifts the 4KB or 8KB command-line length
limitation imposed by the Windows shell when a command uses quoted
strings, because we now don't call the shell in that case.
job.c (create_batch_file): Declare the counter of batch files
static, to avoid having 2 jobs using the same file name and
stepping on each other's toes. When all 64K names are used up,
make one more loop looking for slots that became vacant. This
avoids leaving behind temporary batch files in the temporary
directory, which happens frequently on a fast machine when using
parallel builds.
(exec_command) [WINDOWS32]: Use pid2str instead of non-portable
%Id.
main.c (handle_runtime_exceptions): Use %p to print addresses,
to DTRT on both 32-bit and 64-bit hosts. Savannah bug #27809.
job.c (w32_kill, start_job_command, create_batch_file): Use
pid_t for process IDs and intptr_t for the 1st arg of
_open_osfhandle.
function.c (windows32_openpipe): Use pid_t for process IDs and
intptr_t for the 1st arg of _open_osfhandle.
(func_shell): Use pid_t for process IDs.
main.c (main) [WINDOWS32]: Pacify the compiler.
config.h.W32.template (pid_t): Add a definition for 64-bit
Windows builds that don't use GCC.
Savannah bug #27809. Patch by Ozkan Sezer <sezeroz@gmail.com>
backward-incompatible change in the 2008 POSIX specification.
- Add the .SHELLFLAGS variable so people can choose their own shell flags.
- Add tests for this.
- Add documentation for this.
HAVE_DECL_SYS_SIGLIST.
job.c (child_execute_job): Remove __MSDOS__ because MSDOS/DJGPP build does
not use child_execute_job.
variable.c (define_automatic_variables) [__MSDOS__]: Always export the SHELL
environment variable to the child.
Allows the user to reset the prefix character for introducing recipe lines
from the default (tab) to any other single character, and back again.
Also, reworked the manual to consistently use the word "recipe" to describe
the set of commands we use to update a target, instead of the various
phrases used in the past: "commands", "command lines", "command scripts",
etc.
Fix an uninitialized variable.
Add builtin rules for Objective C.
Add a new debug line that shows where the commands that are about to be run
were defined.
comparison functions to always use POSIX strcasecmp(). For non-POSIX
systems that use other functions (strcmpi or stricmp) use a macro to alias
strcasecmp to those. If we can't find any of them (VMS, plus whatever
UNIX doesn't have them) then define our own version in misc.c.
string into the strcache. As a side-effect, many more structure members and
function arguments can/should be declared const.
As mentioned in the changelog, unfortunately measurement shows that this
change does not yet reduce memory. The problem is with secondary expansion:
because of this we store all the prerequisites in the string cache twice.
First we store the prerequisite string after initial expansion but before
secondary expansion, then we store each individual file after secondary
expansion and expand_deps(). I plan to change expand_deps() to be callable
in either context (eval or snap_deps) then have non-second-expansion
targets call expand_deps() during eval, so that we only need to store that
dependency list once.
A few changes from char* to void* where appropriate, and removing of
unnecessary casts.
Much more work on const-ifying the codebase. This round involves some code
changes to make it correct. NOTE!! There will almost certainly be problems
on the non-POSIX ports that will need to be addressed after the const changes
are finished: they will need to be const-ified properly and there may need to
be some changes to allocate memory, etc. as well.
The next (last?) big push for this, still to come, is const-ifying the
filenames in struct file, struct dep, etc. This will allow us to store file
names in the string cache and finally resolve Savannah bug #15182 (make uses
too much memory), among other advantages.
16304, 16468, 16577, 17701, 17880, 16051, 16652, 16698
Plus some from the mailing list.
Imported a patch from Eli to allow Cygwin builds to support DOS-style
pathnames.
- Add more warnings.
- Rename variables that mask out-scope vars with the same name.
- Remove all casts of return values from xmalloc, xrealloc, and alloca.
- Remove casts of the first argument to xrealloc.
- Convert all bcopy/bzero/bcmp invocations to use memcp/memmove/memset/memcmp.
Revert a fix for $? including non-existent files as it shows a bug
in the Linux kernel build. Give them a release to fix this.
Add some changes from Eli Z. for Windows changes.
- Fix handling of special targets like .SUFFIX for VMS insensitive targets.
- Don't make temporary batch files for -n. Make sure batch files are created
in text mode.
follow POSIX backslash/newline conventions.
Use a different method for testing the SHELL variable, which hopefully
will work better on non-UNIX systems.
