* .gitignore: Exclude /lib/Makefile.am
* bootstrap.conf: Use --makefile-name=gnulib.mk with gnulib-tool
* configure.ac: Use wget_MANYWARNINGS()
* lib/Makefile.am: Define empty noinst_LIBRARIES and MAINTAINERCLEANFILES
* m4/wget_manywarnings.m4: New file from GNU Wget2
* src/Makefile.am: Use WARN_CFLAGS for AM_CFLAGS
* configure.ac: Search for dlopen and add library to $FUZZ_LIBS
* fuzz/Makefile.am: Link with $FUZZ_LIBS instead of -ldl
This fixes linking on BSD systems.
Reported-by: Nam Nguyen
* configure.ac: Check for libpcre2-8
* src/init.c (choices): Test for HAVE_LIBPCRE2
* src/main.c (main): Set regex compile and match functions
* src/options.h: Test for HAVE_LIBPCRE2
* src/utils.c: Include pcre2.h, add functions
compile_pcre2_regex() and match_pcre2_regex()
* src/utils.h: Declare compile_pcre2_regex() and match_pcre2_regex()
Fixes#54677
Reported-by: Noël Köthe
* Makefile.am: Add fuzz/ to SUBDIRS
* cfg.mk: Fix 'make syntax-check'
* configure.ac: Add --enable-fuzzing
* fuzz/Makefile.am: New file
* fuzz/README.md: New file
* fuzz/fuzzer.h: New file
* fuzz/get_all_corpora: New file
* fuzz/get_ossfuzz_corpora: New file
* fuzz/glob_crash.c: New file
* fuzz/main.c: New file
* fuzz/run-afl.sh: New file
* fuzz/run-clang.sh: New file
* fuzz/view-coverage.sh: New file
* fuzz/wget_options_fuzzer.c: New file
* fuzz/wget_options_fuzzer.dict: New file
* src/init.c (cleanup): Free more resources
* src/main.c (init_switches): Initialize only once,
(print_usage): Don't print if TESTING is defined
* src/utils.h: Include wget.h
* configure.ac: Remove checking for IRI=no
On certain systems, gnulib falls back to libunistring
for mbtowc() and wcwidth() (used in src/progress.c).
Reported-by: Zhiming Wang
* .travis.yml: Install libidn2-dev instead libidn11-dev.
* bootstrap.conf: Add modules libunistring-optional, unistr/base,
unicase/tolower.
* configure.ac: Check for libidn2.
* src/Makefile.am: Add $(LTLIBUNISTRING) to LDADD.
* tests/Makefile.am: Set LDADD similar to LDADD in src/Makefile.am
* src/connect.c: Use libidn2 code instead of libidn.
* src/host.c: Likewise.
* src/iri.c: Likewise.
* src/iri.h: Likewise.
* src/options.h: Likewise.
* src/url.c: Likewise.
* src/url.h: Likewise.
* src/log.c: Fix C99 comment.
IDN2003 should not be used any more due to security concerns.
We use libunistring (resp. the unicode code from gnulib) for
lowercasing UTF-8 before we give data to libidn2.
TR#46 is missing, no support in libidn2 nor in libunistring.
* configure.ac: Add --with-psl-file to set a PSL file
* src/cookies.c (check_domain_match): Load PSL_FILE with
fallback to built-in data.
This change allows package maintainers to make Wget use the latest
PSL (DAFSA or plain text), without updating libpsl itself.
E.g. Debian now comes with a DAFSA binary within the 'publicsuffix'
package which allows very fast loading (no parsing or processing needed).
* configure.ac: Check for xattr availability
* src/Makefile.am: Add xattr.c
* src/ftp.c: Include xattr.h.
(getftp): Set attributes if enabled.
* src/http.c: Include xattr.h.
(gethttp): Add parameter 'original_url',
set attributes if enabled.
(http_loop): Add 'original_url' to call of gethttp().
* src/init.c: Add new option --xattr.
* src/main.c: Add new option --xattr, add description to help text.
* src/options.h: Add new config member 'enable_xattr'.
* src/xatrr.c: New file.
* src/xattr.h: New file.
These attributes provide a lightweight method of later determining
where a file was downloaded from.
This patch changes:
* autoconf detects whether extended attributes are available and
enables the code if they are.
* The new flags --xattr and --no-xattr control whether xattr is enabled.
* The new command "xattr = (on|off)" can be used in ~/.wgetrc or /etc/wgetrc
* The original and redirected URLs are recorded as shown below.
* This works for both single fetches and recursive mode.
The attributes that are set are:
user.xdg.origin.url: The URL that the content was fetched from.
user.xdg.referrer.url: The URL that was originally requested.
Here is an example, where http://archive.org redirects to https://archive.org:
$ wget --xattr http://archive.org
...
$ getfattr -d index.html
user.xdg.origin.url="https://archive.org/"
user.xdg.referrer.url="http://archive.org/"
These attributes were chosen based on those stored by Google Chrome
https://bugs.chromium.org/p/chromium/issues/detail?id=45903
and curl https://github.com/curl/curl/blob/master/src/tool_xattr.c
* README.checkout: Add description for libares
* configure.ac: Add check for libares
* doc/wget.texi: Add docs for the new options
* src/build_info.c.in: Add +/-cares for --version output
* src/host.c:
(merge_address_lists): New static function
(address_list_from_hostent): New static function
(wait_ares): New static function
(callback): New static function
(lookup_host): Add libares resolver code
* src/init.c: Add new options,
(cleanup): Add cleanup code
* src/main.c: Add global libares channel variable
(cmdline_option option_data): Add new options
(print_help): Add short descriptions
(main): Add libares init code
* src/options.h (struct options): Add option members
The new options allow to specify alternative DNS servers and
an alternate packet route for the resolver packets.
Wget has to built with libares, enabled at configure time by
./configure --with-cares.
* configure.ac: Export WITH_SSL for use in Makefile.am
* testenv/Makefile.am: Add HTTPS tests to XFAIL_TESTS when !WITH_SSL
Reported-by: Ander Juaristi <ajuaristi@gmx.es>
Make --with-libssl-prefix and --with-libgnutls-prefix do the right thing,
no matter if pkg-config is installed or not.
Reported-by: Charles Diza <chdiza@gmail.com>
--with-libuuid now explictly asks for libuuid.
--without-libuuid ignores libuuid and tries to use libc builtin functions.
Else try builtin functions first, libuuid second and fallback to Wget's own
function.
* Always attempt to detect uuid.h and uuid_create().
* Split libuuid and uuid.h implementations of warc_uuid_str(), since
those APIs vary significantly.
* Correctly use the uuid.h functions
After the Spring Cleaning on warnings in early 2014, GCC and Clang emit
only about two warnings when compiling Wget with -Wall -Wextra.
Enable both these flags by default so that new warnings in these areas
are more prominent.
Adds a --disable-assert configure option. We **highly recommend** the
use of this option by distro maintainers so as to ensure a smooth
experience with Wget. Sometimes an assertion may fail even though Wget
continues to work perfectly. In such scenarios, we do not wish for
client installations of Wget to crash. Hence, for a stable experience,
please use the --disable-assert configure flag.
