* job.c (construct_command_argv_internal): If shell wildcard
characters are found inside a string quoted with "..", give up the
fast route and go through the shell. Fixes Savannah bug #44348.
This fix required a complete rewrite of the command parser vmsjobs.c
child_execute_job. The old parser had too many incorrect assumptions
about DCL commands and could not be repaired to extended.
The parser now more closely parses VMS commands and handles quoted
commands and redirection. Command File mode has been improved, but can
not fully support bs-nl syntax.
VMS Unix shell simulation has been improved.
* commands.c: vms_comma_separator is now a run-time setting.
* function.c: vms_comma_separator is now a run-time setting.
* function.c(func_basename_dir) now reports "[]" or "./" based on
VMS crtl runtime setting.
* job.c(start_job_command): VMS Handle empty commands propery.
* main.c: Add VMS environment variables for run-time settings.
* vms_legacy_behavior - Force older behavior.
* vms_comma_separator - Commas or spaces for separators.
* vms_unix_simulation - Enhanced Posix shell simulation features.
* Detect if VMS CRTL is set to report Unix paths instead of VMS.
* ':' and '>' are also MAP_DIRSEP on VMS.
* makeint.h: Add VMS run-time option variables.
* readme.vms: Update to current behavior.
* variable.c(define_variable_in_set): Fix VMS Environment variable
lookup.
* variable.c(define_automatic_variables): Remove some VMS specific
automatic variables and use the Unix ones instead.
* vms_export_symbol.c: Set max symbol size correctly.
* vmsjobs.c: child_execute_job() complete rewrite of VMS comand
parsing.
* vmsjobs.c(build_vms_cmd): VMS commmand building with shell simulation.
Signed-off-by: Paul Smith <psmith@gnu.org>
This also includes fixing the most of the exit handling code for VMS.
Self tests:
Previously about 94 Tests in 36 categories fail.
Now about 45 tests in 22 categories fail.
Because some tests do not properly clean up, the number of tests that
fail can vary by one or two test cases between consecutive runs.
* Makefile.am: Add new VMS files.
* job.c: add prototype for vms_strsignal().
* job.c: (child_error): Remove VMS specific code as no longer needed.
* job.c: (reap_children): The VMS specific code was setting the
status to 0 instead of setting it to the proper exit status.
* job.h: Add vms_launch_status to struct child.
* main.c: (main): Use environment variables for options to use MCR
* instead of a foreign command, and to always use command files for
subprocesses.
For VMS use (set_program_name) routine which is common to ports of
other GNU packages to VMS to set the program name used internally.
Use (vms_putenv_symbol) to set up symbols to be visible in child
programs, including recursive make launched by execve()
Start of Bash shell detection code for VMS.
* makefile.com: Need nested_include=none for building on VMS search
lists. Add vms_progname, vms_exit, and vms_export_symbol.
* makefile.vms: Need nested_include=none for building on VMS search
lists. Add vms_progname, vms_exit, vms_export_symbol.
* makeint.h: Make sure non-standard "VMS" macro is defined. Add
prototypes for new VMS routines. Remove VMS-specific failure codes.
* vmsjobs.c: Add VMS POSIX exit code constants.
(_is_unixy_shell): Detect Bash shell.
(vms_strsignal): simulate strsignal() on VMS.
(vmsHandleChildTerm): fix to properly report failed LIB$SPAWN() exit
status codes. Remove code that duplicated code in job.c.
(child_execute_job): Export environment symbols before spawning a
child and restore afterward unless option to use command files for
subprocesses is set. Improve handling of UNIX null commands ":".
* vms_exit.c: Provides vms_exit() to detect if an exit code is UNIX
or VMS, and converts the UNIX code into a VMS exit code.
* vms_export_symbol.c: Routines to create DCL symbols that work like
shell aliases or exported shell symbols and clean them up on exit.
* vms_export_symbol_test.com: Unit test for vms_export_symbol.c
* vms_progname.c: New file: VMS specific replace for progname.c that
is used in some GNU projects.
* main.c (find_and_set_default_shell, main) [WINDOWS32]: Declare
variables 'const char *' to avoid compiler warnings.
* job.c (construct_command_argv_internal) [!NDEBUG]: Declare 'end'
and set it only if NDEBUG is not defined, to avoid compiler
warnings.
* job.c (start_job_command) [WINDOWS32]: Compute outfd and errfd
as on Posix platforms, and pass the results to process_easy.
* function.c (windows32_openpipe) [WINDOWS32]: Accept an
additional argument ERRFD and use it for redirecting the standard
error handle passed to the subprocess.
(func_shell_base) [WINDOWS32]: Pass the computed errfd to
windows32_openpipe.
* job.c, vmsjobs.c: fix some double quote and new line handling;
implement ONESHELL with writing multiple lines into one DCL command
procedure; in ONESHELL allow VMS/make internal redirection only on the
first line; fix the created DCL command procedure, which didn't abort
on errors; return correct exit status from the DCL command procedure;
preserve current procedure verification; make the generated command
procedure more robust.
* commands.c, function.c, hash.c, job.c, main.c, output.c:
use MAKE exit codes.
* makeint.h: encode make exit codes so that they are VMS compatible.
* job.c: check child exit code for VMS style exit codes.
* vmsjobs.c: save and return VMS style exit code.
* job.c: split the command line at a newline.
* default.c, vmsjobs.c: change ECHO variable to a pseudo builtin,
which ensures that the VMS/DCL ECHO ("write sys$output") is used
and is correctly quoted.
* vmsjobs.c: remove unused builtin 'rm'.
job.c (start_job_command): Move the child output diversion out
of non-EMX branch.
[__EMX__]: Don't use fixed FD_STDOUT and FD_STDERR in the call to
child_execute_job.
Copyright-paperwork-exempt: yes
* main.c (main): Use ONS(), not OSN().
(prepare_mutex_handle_string) [WINDOWS32]: Use %Ix formatting to
support both 32bit and 64bit systems.
* job.c (free_child, new_job): Use ONS(), not OSN().
* w32/subproc/w32err.c (map_windws32_error_to_string): Use O() when
calling fatal().
Copyright-paperwork-exempt: yes
* output.c (error, fatal, message): Take an extra argument specifying
how many bytes are used by the formatted arguments.
(get_buffer): New function that allocates the requested buffer size.
Remove msc_vsnprintf(), vfmtconcat(), and fmtconcat() as unneeded.
* makeint.h: Declare various helper macros for generating output.
* *.c: Change all error(), fatal(), message() calls to use the macros,
or pass the extra length argument directly.
In this mode we still collect all the output from a given target and
dump it at once. However we don't treat recursive lines any differently
from non-recursive lines. Also we don't print enter/leave messages
after every dump. However we do ensure that we always print them once
to stdout, so the parent make will collect it properly.
Create a new file, output.c, and collect functions that generate output there.
We introduce a new global context specifying where output should go (to stdout
or to a sync file), and the lowest level output generator chooses where to
write output based on that context.
This allows us to set the context globally, and all operations that write
output (including functions like $(info ...) etc.) will use it.
Removed the "--trace=dir" capability. It was too confusing. If you have
directory tracking enabled then output sync will print the enter/leave message
for each synchronized block. If you don't want that, disable directory
tracking.
In various places we were passing flags and characters to compare, then
using complex conditionals to see where to stop in string searches.
Performance numbers reveal that we were spending as much as 23% of our
processing time in these functions, most of it in the comparison lines.
Instead create a character map and use a single bitwise comparison to
determine if this is any one of the stop characters.
This mode replaces the previous heuristic setting enabled with -O, where we
would log directory enter/leave for each synchronized output. Now we only
do that if --trace=dir is given.
job.c (child_out): Output the newline following the message
before fllush-ing the stream. Avoids displaying the following
failure message, which goes to stderr, on the same line.
GCC was giving us warnings, most OS's now just run fork() when you call
vfork(), and looking at the standard definition of vfork() we are a long way
from using it safely anyway: you're not allowed to even call a function before
you exec().
If output-sync is enabled, have make write the command line to the temp file
instead of printing it directly to the screen to ensure that the output is
ordered properly. Also, remove extraneous enter/leave operations by having
them printed directly when dumping temp file output.
If we are not going to sync a command line then dump any collected output
first to preserve ordering. Do some code cleanup:
* Move the handle init to a separate function.
* Move the temp file truncation to the output function.
* Remember whether we sync in a variable for readability.
* Handle EINTR and short writes in child_out().
* Always call sync_output() in case output_sync was changed due to error.
job.c (start_job_command): Make the condition for creating a
temporary output file be identical to the Posix code branch.
Suggested by Frank Heckenbach <f.heckenbach@fh-soft.de>.
job.c (construct_command_argv_internal) [WINDOWS32]: Return
right after generating new_argv for one_shell case. This fixes
the Windows build for both Unixy shell and stock Windows shells.
Enhance the child_error() function so that it will write error output to the
child's sync output buffer, if it exists. If it doesn't the output goes to
stdout/stderr.
A new flag to the -O/--output-sync, "job", selects a per-job (that is, per
line of a recipe) output synchronization. To support this move the close of
the temp file out of the sync_output() function and don't do it until we free
the child, since we may call sync_output() multiple times in a given recipe.
When we set up for a new temp file, if we're in per-job mode we truncate the
file and seek to the beginning to re-use it for every job.
