* configure.ac: Add --with-psl-file to set a PSL file
* src/cookies.c (check_domain_match): Load PSL_FILE with
fallback to built-in data.
This change allows package maintainers to make Wget use the latest
PSL (DAFSA or plain text), without updating libpsl itself.
E.g. Debian now comes with a DAFSA binary within the 'publicsuffix'
package which allows very fast loading (no parsing or processing needed).
* src/cookies.c (cookie_header): Use heap instead of stack.
* src/http.c (request_send): Likewise.
If wget has to handle an insanely large amount of cookies (~700,000 on
32 bit systems or ~530,000 on 64 bit systems), the stack is not large
enough to hold these pointers, leading to undefined behaviour according
to POSIX; expect a segmentation fault in real life. ;)
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The signal handler for SIGALRM calls longjmp, but the handler is
installed before the jump target has been initialized. If another
process sends SIGALRM right between handler installation and target
initialization, the jump leads to undefined behavior.
This can easily be fixed by moving the signal handler installation
into the "SETJMP == 0" conditional block, which means that the target
has just been initialized.
* src/utils.c: call signal after SETJMP.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
* src/init.c: Remove hyphens from command names
* src/main.c: Likewise
Options with hyphens (or underscores) in their command name cannot be
set in a wgetrc file.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* src/metalink.c (retrieve_from_metalink): Continue file download if
opt.always_rest is true
Without --continue, download as a new file with an unique name (this
conforms to the old behaviour).
* bootstrap.conf: Add crypto/md2, and crypto/md4
* src/metalink.c (retrieve_from_metalink): Add md2, and md4 support
This patch adds support for the deprecated (insecure) md2, and md4
Message-Digest algorithms to the Metalink module.
* bootstrap.conf: Add crypto/sha512
* src/metalink.c (retrieve_from_metalink): Add md5, sha1, sha224,
sha384, and sha512 support
Metalink's checksum verification was limited to sha256. This patch
adds support for md5, sha1, sha224, sha384, and sha512.
* configure.ac: Check for xattr availability
* src/Makefile.am: Add xattr.c
* src/ftp.c: Include xattr.h.
(getftp): Set attributes if enabled.
* src/http.c: Include xattr.h.
(gethttp): Add parameter 'original_url',
set attributes if enabled.
(http_loop): Add 'original_url' to call of gethttp().
* src/init.c: Add new option --xattr.
* src/main.c: Add new option --xattr, add description to help text.
* src/options.h: Add new config member 'enable_xattr'.
* src/xatrr.c: New file.
* src/xattr.h: New file.
These attributes provide a lightweight method of later determining
where a file was downloaded from.
This patch changes:
* autoconf detects whether extended attributes are available and
enables the code if they are.
* The new flags --xattr and --no-xattr control whether xattr is enabled.
* The new command "xattr = (on|off)" can be used in ~/.wgetrc or /etc/wgetrc
* The original and redirected URLs are recorded as shown below.
* This works for both single fetches and recursive mode.
The attributes that are set are:
user.xdg.origin.url: The URL that the content was fetched from.
user.xdg.referrer.url: The URL that was originally requested.
Here is an example, where http://archive.org redirects to https://archive.org:
$ wget --xattr http://archive.org
...
$ getfattr -d index.html
user.xdg.origin.url="https://archive.org/"
user.xdg.referrer.url="http://archive.org/"
These attributes were chosen based on those stored by Google Chrome
https://bugs.chromium.org/p/chromium/issues/detail?id=45903
and curl https://github.com/curl/curl/blob/master/src/tool_xattr.c
* src/openssl.c (ssl_init): Use SSL_is_init_finished() instead of
SSL_state(), conditionally skip SSLeay function calls
The python test suite makes SSL_peek() hang, consuming 100% CPU time.
This does not happen on real world TLS connections, though, but needs
investigations.
* src/hsts.c (hsts_file_access_valid): we should check for "world-writable"
files only on Unix-based systems. It's difficult to mimic the same behavior
on Windows, so it's better to just not do it.
Reported-by: Gisle Vanem <gvanem@yahoo.no>
Reported-by: Eli Zaretskii <eliz@gnu.org>
If not --trust-server-names is used, FTP will also get the destination
file name from the original url specified by the user instead of the
redirected url. Closes CVE-2016-4971.
* src/ftp.c (ftp_get_listing): Add argument original_url.
(getftp): Likewise.
(ftp_loop_internal): Likewise. Use original_url to generate the
file name if --trust-server-names is not provided.
(ftp_retrieve_glob): Likewise.
(ftp_loop): Likewise.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* src/main.c (save_hsts): save the in-memory HSTS database to a file
only if something changed.
* src/hsts.c (struct hsts_store): new field 'changed'.
(hsts_match): update field 'changed' accordingly.
(hsts_store_entry): update field 'changed' accordingly.
(hsts_store_has_changed): new function.
* src/hsts.h (hsts_store_has_changed): new function.
* hsts.c (hsts_file_access_valid): check that the file is a regular
file, and that it's not world-writable.
(hsts_store_open): if the HSTS database file does not meet the
above requirements, disable HSTS at all.
* src/hsts.c (hsts_store_entry): strictly comply with RFC 6797.
RFC 6797 states in section 8.1 that the UA's cached information should
only be updated if:
"either or both of the max-age and includeSubDomains header field
value tokens are conveying information different than that already
maintained by the UA."
* src/metalink.c (retrieve_from_metalink): Fix debug message to print the
fingerprint instead of a pointer.
* Do not delete the ChangeLog file since it is required by the Makefile
and breaks compilation
* README.checkout: Add description for libares
* configure.ac: Add check for libares
* doc/wget.texi: Add docs for the new options
* src/build_info.c.in: Add +/-cares for --version output
* src/host.c:
(merge_address_lists): New static function
(address_list_from_hostent): New static function
(wait_ares): New static function
(callback): New static function
(lookup_host): Add libares resolver code
* src/init.c: Add new options,
(cleanup): Add cleanup code
* src/main.c: Add global libares channel variable
(cmdline_option option_data): Add new options
(print_help): Add short descriptions
(main): Add libares init code
* src/options.h (struct options): Add option members
The new options allow to specify alternative DNS servers and
an alternate packet route for the resolver packets.
Wget has to built with libares, enabled at configure time by
./configure --with-cares.
* src/gnutls.c (ssl_connect_wget, ssl_check_certificate): Fix SNI server name
* src/openssl.c (ssl_connect_wget, ssl_check_certificate): Fix SNI server name
Fixes#47408
* src/url.c [HAVE_ICONV]: Include iconv.h and langinfo.h.
(convert_fname): New function.
[HAVE_ICONV]: Convert file name from remote encoding to local
encoding.
(url_file_name): Call convert_fname.
(filechr_table): Don't consider bytes in 128..159 as control
characters.
* tests/Test-ftp-iri.px: Fix the expected file name to match the
new file-name recoding. State the remote encoding explicitly on
the Wget command line.
* NEWS: Mention the URI recoding when built with libiconv.
