Rework the single "?=" operator to instead allow a "?" modifier to be
prepended to ANY assignment operator. If "?" is given then the
variable is assigned (using whatever operator comes next) if and only
if the variable is not already defined. If it is defined then no
action is taken (the right-hand side is not expanded, etc.)
* NEWS: Announce this new feature.
* doc/make.texi: Modify the documentation around assignment operators.
* src/variable.h: Remove the f_conditional variable flavor.
(do_variable_definition): Add an argument specifying conditional.
* src/variable.c (parse_variable_definition): Use the existing flag
"conditional" to remember if we saw "?" rather than the flavor.
When we see "?" skip it and continue trying to parse an assignment.
(try_variable_definition): Pass condition to do_variable_definition().
(initialize_file_variables): Ditto.
(do_variable_definition): Check for conditional up-front: quit if set.
Remove handling of obsolete f_conditional flavor.
* src/read.c (eval_makefile): MAKEFILE_LIST is not conditional.
(do_define): Unset conditional for define with no operator. Pass the
conditional flag to do_variable_definition().
(construct_include_path): .INCLUDE_DIRS is not conditional.
* src/load.c (load_file): .LOADED is not conditional.
* tests/scripts/variables/conditional: Add new tests.
* README.git: Add some notes about using ASAN.
* src/makeint.h: Declare skip_references().
* src/misc.c (skip_reference): A new function that will skip over a
variable reference, counting matching open paren/brace characters.
* src/implicit.c (get_next_word): Replace code with skip_reference().
* src/read.c (conditional_line): Ditto.
(find_map_unquote): Ditto.
(get_next_mword): Ditto.
(parse_variable_definition): Ditto.
* src/function.c (handle_function): Make clear that the passed in
pointers are not modified if the function returns false.
* src/expand.c (expand_string_buf): Don't create local variables to
call handle_function() since it doesn't modify its arguments.
* src/job.c (new_job): Small simplifications.
Create a new special variable, .WARNINGS, to allow per-makefile
control over warnings. The command line settings will override
this.
Move the handling of warning flags to a new file: src/warning.c.
Allow the decode to work with generic strings, and call it from
decode_switches().
* Makefile.am: Add new file src/warning.c.
* build_w32.bat: Ditto.
* builddos.bat: Ditto.
* po/POTFILES.in: Ditto.
* src/makeint.h: #define for the .WARNINGS variable name.
* src/warning.h: Add declarations for methods moved from main.c.
Rename the enum warning_state to warning_action.
* src/warning.c: New file. Move all warning encode/decode here
from main.c.
* src/main.c: Move methods into warning.c and call those methods
instead.
(main): Set .WARNINGS as a special variable.
* src/job.c (construct_command_argv): Rename to warning_action.
* src/read.c (tilde_expand): Ditto.
* src/variable.c (set_special_var): Update warnings when the
.WARNINGS special variable is set.
* tests/scripts/options/warn: Check invalid warning options.
* tests/scripts/variables/WARNINGS: Add tests for the .WARNINGS
special variable.
The "invalid-var" warning triggers if the makefile attempts to assign
a value to an invalid variable name (a name containing whitespace).
The "invalid-ref" warning triggers if the makefile attempts to
reference an invalid variable name. Both new warnings have a default
action of "warn".
* NEWS: Add these new warnings.
* doc/make.1: Document them in the man page.
* doc/make.texi (Warnings): Document them in the user's manual.
* src/warning.h: Add enum values for the new warning types.
* src/main.c (initialize_warnings): Initialize the new warnings.
* src/variable.h (undefine_variable_in_set, undefine_variable_global):
Ask callers to provide a struct floc specifying where the variable
is undefined.
* src/read.c (do_undefine): Pass floc when undefining.
* src/variable.c (check_valid_name): If invalid-var is enabled, check
the variable name.
(define_variable_in_set): Call it.
(undefine_variable_in_set): Ditto.
(check_variable_reference): If invalid-ref is enabled, check the
variable reference.
(lookup_variable): Call it.
(lookup_variable_in_set): Ditto.
* tests/scripts/options/warn: Add tests for the new warning types.
Replace the singleton --warn-undefined-variables with infrastructure
to manage multiple warnings: the --warn option can take an action
"ignore", "warn", or "error" (which will apply to all warnings), or
a specific warning type and an action for that type. Multiple
options can be provided and are consolidated.
* NEWS: Announce the new option.
* doc/make.1: Document in the man page.
* doc/make.texi (Warnings): Document in the user's manual.
* Makefile.am: Add new header warning.h.
* src/warning.h: Define enum for actions and warning types, and
macros to test whether they are set. Keep the default settings
separate so that we can correctly reconstruct MAKEFLAGS.
* src/makeint.h: Remove deprecated warn_undefined_variables_flag.
* src/main.c: Create global variables to hold warning settings.
(switches): Add a new switch for --warn.
(initialize_warnings): Set the default warning actions.
(main): Call initialize_warnings().
(encode_warning_state, decode_warning_state): Convert warning states
between strings and enums.
(encode_warning_name, decode_warning_name): Convert warning names
between strings and enums.
(decode_warn_flags): Convert a --warn option into enum values. If
deprecated warn_undefined_variables_flag is set convert it to --warn.
(decode_switches): Don't remove duplicates of --warn since order
matters. Call decode_warn_flags() to handle --warn.
(define_makeflags): Special-case handling of --warn options written
to MAKEFLAGS: write out the current settings.
* src/read.c (tilde_expand): Use new warning control macros.
* src/variable.c (warn_undefined): Ditto.
* src/job.c (construct_command_argv): Ditto.
* tests/scripts/options/warn: Rename from warn-undefined-variables
and add tests for --warn.
* tests/scripts/variables/MAKEFLAGS: Expect the new behavior.
Return the current buffer instead of freeing it.
(variable_append): Use install/swap to handle variable buffers.
(allocated_variable_expand_for_file): Ditto.
* src/variable.c (shell_result): Ditto.
* src/variable.h: Declare the new function.
* src/expand.c (variable_buffer_output): Allocate an extra byte and
set it to nul. For safety, add assert() on PTR.
(variable_expand_string): Don't call variable_buffer_output just to
add a nul byte.
(allocated_variable_append): Ditto.
* configure.ac: Set MK_OS_W32 to 1 if we're on Windows32.
* src/config.h.W32: Ditto.
* mk/Windows32.mk: Ditto.
* src/makeint.h: Convert #if refs to WINDOWS32, to use MK_OS_W32.
* src/*: Ditto.
* src/w32/*: Ditto.
* gl/lib/*: Ditto.
* src/variable.c (lookup_variable_for_file): New function to retrieve
a variable assignment in a file context.
* src/variable.h (lookup_variable_for_file): Declare it.
* src/job.c (construct_command_argv): Look up .SHELLFLAGS. If .POSIX
is set and we're using the default value, choose -c if we're ignoring
errors else choose -ec.
(construct_command_argv_internal): Ditto.
* tests/scripts/targets/POSIX: Add tests.
If -e was given we weren't expanding MAKEFLAGS before passing it
through the environment to jobs: we don't expand variables we
receive from the environment and when -e is given we set the
origin of MAKEFLAGS to "environment override". Check for MAKEFLAGS
specifically, which seems like a hack but I don't have a better
idea offhand.
* src/main.c (main): Drive-by: use o_default for MAKEOVERRIDES.
* src/variable.c (target_environment): Always expand MAKEFLAGS
regardless of the origin type.
* tests/scripts/options/dash-e: Create a test.
Reported by Dmitry Goncharov <dgoncharov@users.sf.net>
* src/variable.c (struct defined_vars): Create a struct that holds the
name and length of each variable name.
(warn_undefined): Check the lengths before comparing the contents.
* tests/scripts/options/warn-undefined-variables: Add a test.
If a parent target has an exported variable that is private, don't
export it in child targets.
* NEWS: Mention this change.
* src/variable.c (target_environment): Ignore private inherited
variables.
* tests/thelp.pl: Add a new "env" operation to show env.var. values.
* tests/scripts/variables/private: Verify this new behavior.
Don't generate undefined variable warnings for variables that are
internal / special to make and where the empty string is valid.
Rather than defining them to empty, which could introduce unwanted
behavior, keep a list of variable names which we should never warn
about.
* src/variable.h (warn_undefined): Convert the macro to a function.
* src/variable.c (defined_vars): Always "defined" variable names.
(warn_undefined): Implement as a function and check against the
defined_vars before generating a warning.
* src/read.c (read_all_makefiles): No need to reset warning flag.
* src/vpath.c (build_vpath_lists): Ditto.
* tests/scripts/options/warn-undefined-variables: Expand all the
pre-defined variables to ensure warnings are not generated.
* src/makeint.h (reset_makeflags): New function to handle changing
MAKEFLAGS from within makefiles. Remove decode_env_switches().
* src/variable.c (set_special_var): Call reset_makeflags() instead
of various internal methods.
* src/main.c (decode_env_switches): Only internal now so make static.
(decode_switches): Don't invoke construct_include_path() yet.
(reset_makeflags): Decode env switches and construct include paths.
(main): Construct include paths after we process -C options.
* tests/scripts/options/dash-C: Rewrite to use new test constructs.
Add a test using both -C and -I together.
Add a test for multiple -C options.
This commit introduces two visible changes:
1. Keep command line variable assignments in MAKEFLAGS at all times,
even while parsing makefiles.
2. Define makeflags immediately when a makefile modifies MAKEFLAGS.
The new MAKEFLAGS and MAKEOVERRIDES initialization procedure:
1. decode_switches (argc, argv, o_command) is called to parse command
line variable assignments.
2. Command line variable assignments go through quote_for_env.
Initialize -*-command-variables-*- to the quoted values.
3. MAKEOVERRIDES is initialized to refer to -*-command-variables-*-
with origin o_env to keep the definitions in the database intact.
4. define_makeflags() is called which adds MAKEOVERRIDES to MAKEFLAGS.
5. Makefiles are parsed. If a makefile modifies MAKEFLAGS, the new
value of MAKEFLAGS is defined right away.
6. Env switches are decoded again as o_env. The definitions set by
decode_switches at step 1 stay intact, as o_command beats o_env.
We must preserve the original intact definitions in order to detect
failure cases; for example:
$ cat makefile
all:; $(hello)
$ make hello='$(world'
makefile:1: *** unterminated variable reference. Stop.
* src/makeint.h: Declare enum variable_origin, struct variable and
define_makeflags(). Add parameter origin to decode_env_switches().
* src/main.c (define_makeflags): Remove "all". If a variable is
assigned on the command line then append MAKEOVERRIDES to MAKEFLAGS.
(decode_env_switches): Replace parameter env with origin.
(decode_switches): Replace parameter env with origin.
Treat origin == o_command as env == 0.
(handle_non_switch_argument): Replace parameter env with origin.
Treat origin == o_command as env == 0.
(main): Call decode_switches() with origin==o_command before parsing
makefiles. Call decode_switches() with origin==o_env after parsing
makefiles.
* src/variable.c (set_special_var): Define makeflags at parse time,
each time a makefile modifies MAKEFLAGS.
(do_variable_definition): Strip command line variable assignments from
MAKEFLAGS before appending extra flags. set_special_var() adds them
back.
* tests/scripts/variables/MAKEFLAGS: Add tests.
The fix for SV 10593 caused recursive expansion errors when exporting
a variable that contains a $(shell ...) invocation. If we see this
type of recursion, ignore it and expand to the empty string rather
than failing.
* src/variable.h (env_recursion): New global variable.
* src/variable.c (target_environment): If creating the environment
for a $(shell ...) function increment env_recursion. Remove the
check for expansion in a shell function context.
* src/expand.c (recursively_expand_for_file): Check for recursive
expansion in a $(shell ...) environment context and if present,
show the verbose message and return the empty string.
* tests/scripts/functions/shell: Add a test for this situation.
Savannah issues such as SV 57242 and SV 62397 show how passing
references to closed file descriptors via the --jobserver-auth option
in MAKEFLAGS can lead to problematic outcomes.
When computing the child environment for a non-recursive shell, add
an extra option to MAKEFLAGS to disable the file descriptors for the
jobserver.
Unfortunately this doesn't modify the value of the make variable
MAKEFLAGS, it only modifies the value of the sub-shell environment
variable MAKEFLAGS. This can lead to confusion if the user is not
considering the distinction.
* src/makeint.h: Publish the jobserver-auth value. Add a global
definition of the name of the command line option.
* src/os.h (jobserver_get_invalid_auth): New function to return a
string invalidating the jobserver-auth option.
* src/w32/w32os.c (jobserver_get_invaid_auth): Implement it. On
Windows we use a semaphore so there's no need to invalidate.
* src/posixos.c (jobserver_parse_auth): If we parse the invalid
auth string, don't set up the jobserver.
(jobserver_get_invalid_auth): Return an invalid option.
* src/variable.h (target_environment): Specify if the target
environment is for a recursive shell or non-recursive shell.
* src/variable.c (target_environment): Move checking for MAKELEVEL
into the loop rather than doing it at the end.
Along with this, check for MAKEFLAGS and MFLAGS, and update them
based on whether we're invoking a recursive or non-recursive child,
and also on whether it's necessary to invalidate the jobserver.
* src/function.c (func_shell_base): Shell functions can never be
recursive to pass 0 to target_environment().
* src/job.c (start_job_command): Specify whether the child is
recursive when calling target_environment().
* src/main.c: Export jobserver_auth. sync_mutex doesn't need to
be exported. Use the global definition for the option name.
* tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
* src/makeint.h (stpcpy): Add missing declaration.
* src/amiga.c (MyExecute): Avoid extra strlen using stpcpy.
* src/function.c (func_shell_base): Ditto.
(func_error): Use memcpy() not strcpy() when we know the length.
* src/job.c (construct_command_argv_internal): Use stpcpy().
* src/main.c (main): Ditto.
(define_makeflags): Ditto.
* src/variable.c (print_target_variables): Use memcpy() when we know
the length.
* src/commands.c (set_file_variables): Use mempcpy().
* src/expand.c (variable_buffer_output): Ditto.
* src/file.c (expand_deps): Ditto.
* src/function.c (abspath): Ditto.
(handle_function): Ditto.
* src/implicit.c (pattern_search): Ditto.
* src/job.c (construct_command_argv_internal): Use mempcpy() and
don't add multiple spaces when there are no shell flags.
* src/main.c (decode_env_switches): Use mempcpy() to simplify.
(define_makeflags): Ditto.
* src/variable.c (selective_vpath_search): Ditto.
Export all variables, including exported makefile variables, when
invoking a shell for the $(shell ...) function. If we detect a
recursive variable expansion, silently ignore that variable and do
not export it. We do print a debug message.
* NEWS: Announce the potential backward-incompatibility.
* doc/make.texi (Shell Function): Document the export behavior.
* src/main.c (main): Add "shell-export" to .FEATURES.
* src/job.h: New function to free struct childbase.
* src/job.c (free_childbase): Implement it; call from free_child.
* src/function.c (func_shell_base): Use target_environment() to
obtain the proper environment for the shell function.
Use free_childbase() to free memory.
(windows32_openpipe): Don't reset the environment: the caller
already provided a proper PATH variable in envp.
* src/variable.c (target_environment): If we detect a recursive
expansion and we're called from func_shell, ignore the variable.
(sync_Path_environment): Simplify and reduce memory allocation.
* tests/scripts/functions/shell: Add tests for this.
Commit f2771aa614 introduced a bug where some switches were left out
of MAKEFLAGS. Instead of resetting switches, get the same results by
filtering out duplicates.
* src/makeint.h: Remove reset_switches.
* src/main.c: (reset_switches): Remove reset_switches.
* (main): Remove call to reset_switches.
* (decode_switches): Filter out duplicate flags.
* src/variable.c: (set_special_var): Remove call to reset_switches.
* tests/scripts/variables/MAKEFLAGS: Verify that duplicate flags are
properly filtered out.
* src/main.c (main): Use a separate variable to track final character.
* src/read.c (eval): Track the semicolon position not one beyond it.
* src/variable.c (do_variable_definition): Include a default switch
case to ease the work of the exhaustiveness prover.
POSIX Issue 8 will require a new assignment operator, :::=.
This operator behaves similarly to the BSD make := operator: the
right-hand side is expanded immediately, but then the value is
re-escaped (all '$' are converted to '$$') and the resulting variable
is considered a recursive variable: the value is re-expanded on use.
* src/variable.h (enum variable_flavor): Add f_expand flavor.
* src/variable.c (do_variable_definition): When defining f_expand,
post-process the result to re-escape '$' characters.
Remove default: to the compiler warns about un-handled enum values.
Set recursive values for both f_recursive and f_expand.
(parse_variable_definition): Rewrite this method.
The previous version was annoying to extend to ':::='.
(print_variable): Remove default: so the compiler warns us about
un-handled enum values.
* src/function.c (func_origin): Remove default: so the compiler warns
us about un-handled enum values.
* doc/make.texi: Add documentation for :::=.
* tests/scripts/variables/define: Add a test for define :::=.
* tests/scripts/variables/flavors: Add tests for :::=.
* tests/scripts/variables/negative: Add tests for :::=.
When MAKEFLAGS is set in a makefile, reparse it immediately rather
than waiting until after all makefiles have been read and parsed.
This change doesn't actually fix the SV bug referenced because, even
though we do reparse MAKEFLAGS, we don't handle the -r or -R options
immediately. Doing this will require more effort.
* NEWS: Announce the change.
* src/makeint.h: Publish reset_switches() and decode_env_switches()
from main.c
* src/main.c (main): Don't call construct_include_path(); it will be
invoked decode_switches().
Preserve the old values of builtin_rules, builtin_variables, and
job_slots before we read makefiles since they can be changed now.
(reset_switches): Publish (remove static). Set the initial value of
the stringlist list to NULL.
(decode_switches): Call construct_include_path() after decoding.
(decode_env_switches): Publish (remove static).
(define_makeflags): Set the MAKEFLAGS variable for special handling.
* src/read.c (eval_makefile): Check for empty include_directories.
(construct_include_path): Clear any old value of .INCLUDE_DIRS before
appending new values. Free the previous include_directories.
* src/variable.c (lookup_special_var): When MAKEFLAGS is set, first
reset the switches then re-parse the variable.
* tests/run_make_tests.pl: Memo-ize some default variable values.
* tests/scripts/options/dash-r: Create tests for setting -r and -R.
* tests/scripts/variables/MAKEFLAGS: Test that resetting -I from
within the makefile takes effect immediately.
Rewrite the environment variable algorithm to correctly inherit
export settings from parent variable sets. The new algorithm
for computing the table of environment variables is:
- Start with the most local variable set and proceed to global.
- If the variable already exists in the table and we don't know
its export status, update it with the current variable's status.
- If the variable is not in the table and it's not global, add it
regardless of its status so if it's unexported we remember that.
- If the variable is not in the table and is global, check its
export status and don't add it if we won't export it.
Then when generating the environment variables, check the export
status of each variable in case it was a target-specific variable
and we have determined it should not be exported.
Rework SHELL handling to check at the end whether we added it or
not and if we didn't, add the value from the environment.
* NEWS: Announce support for target-specific "unexport"."
* doc/make.texi (Target-specific): Document the support.
* src/variable.h (enum variable_export): Make into a global type.
* src/read.c (struct vmodifiers): Use enum variable_export rather
than individual booleans.
(parse_var_assignment): Parse the "unexport" keyword.
(eval): Remember the vmodifier value in the variable.
(record_target_var): Ditto.
* src/variable.c (should_export): Check if the variable should be
exported.
(target_environment): Implement the above algorithm.
* tests/scripts/features/export: Test export/unexport with variable
assignments on the same line.
* tests/scripts/features/targetvars: Add a comprehensive suite of
tests for different types of target-specific export / unexport.
* tests/scripts/variables/SHELL: Update the comment.
When checking for invalid environment variable names we searched the
entire name string instead of just the first LENGTH chars; this could
cause us to incorrectly decide the variable was not exportable.
Dmitry Goncharov <dgoncharov@users.sf.net> found this bug and
provided a test case and sample fix: I used the test but chose a
slightly different fix.
* src/variable.c (define_variable_in_set): check the variable name
not the input string.
* tests/scripts/features/targetvars: Ensure environment variable
values are exported.
When merging the variable_set_lists for two targets it could be
that the "from" set list is a subset of the "to" set list: check
for this situation to avoid freeing used memory.
* src/variable.c (merge_variable_set_lists): Walk the "to" list and
if the "from" list is contained in it, nothing to do.
* tests/scripts/features/se_explicit: Add a test.
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.
* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
