Also the Referer info is reduced to scheme+host+port.
* src/ftp.c (getftp): Change params of set_file_metadata()
* src/http.c (gethttp): Change params of set_file_metadata()
* src/xattr.c (set_file_metadata): Remove user/password from origin URL,
reduce Referer value to scheme/host/port.
* src/xattr.h: Change prototype of set_file_metadata()
* src/init.c (defaults): Set enable_xattr to false by default
* src/main.c (print_help): Reverse option logic of --xattr
* doc/wget.texi: Add description for --xattr
Users may not be aware that the origin URL and Referer are saved
including credentials, and possibly access tokens within
the urls.
This reverts commit 6f3b995993.
The code is obviously wrong, see https://savannah.gnu.org/bugs/?54963
Also, the example from the original post doesn't work any more.
With other words, the broken server behavior has been fixed meanwhile.
* src/openssl.c: Check for OPENSSL_NO_ENGINE before
including openssl/engine.h and before calling ENGINE_load_builtin_engines()
Fixes compilation with no engines compiled.
Copyright-paperwork-exempt: Yes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Current sufmatch does not match when domain is dot-prefixed.
The example of no_proxy in man (.mit.edu) does use a dot-prefixed
domain.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
* configure.ac: Check for libpcre2-8
* src/init.c (choices): Test for HAVE_LIBPCRE2
* src/main.c (main): Set regex compile and match functions
* src/options.h: Test for HAVE_LIBPCRE2
* src/utils.c: Include pcre2.h, add functions
compile_pcre2_regex() and match_pcre2_regex()
* src/utils.h: Declare compile_pcre2_regex() and match_pcre2_regex()
Fixes#54677
Reported-by: Noël Köthe
* doc/wget.texi: Add "TLSv1_3" to --secure-protocol
* src/gnutls.c (set_prio_default): Use GNUTLS_TLS1_3 where needed
Wget currently allows specifying "TLSv1_3" as the parameter for
--secure-protocol option. However it is only implemented for OpenSSL
and in case wget is compiled with GnuTLS, it causes wget to abort with:
GnuTLS: unimplemented 'secure-protocol' option value 6
GnuTLS contains TLS 1.3 implementation since version 3.6.3 [1]. However
currently it must be enabled explicitly in the application of it to be
used. This will change after the draft is finalized. [2] However for
the time being, I enabled it explicitly in case "TLSv1_3" is used with
--secure-protocol.
I also fixed man page to contain "TLSv1_3" in all listings of available
parameters for --secure-protocol
[1] https://lists.gnupg.org/pipermail/gnutls-devel/2018-July/008584.html
[2] https://nikmav.blogspot.com/2018/05/gnutls-and-tls-13.html
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Error: RESOURCE_LEAK (CWE-772): - REAL ERROR
wget-1.19.5/src/warc.c:1376: alloc_fn: Storage is returned from allocation function "url_escape".
wget-1.19.5/src/url.c:284:3: alloc_fn: Storage is returned from allocation function "url_escape_1".
wget-1.19.5/src/url.c:255:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:255:3: var_assign: Assigning: "newstr" = "xmalloc(newlen + 1)".
wget-1.19.5/src/url.c:258:3: var_assign: Assigning: "p2" = "newstr".
wget-1.19.5/src/url.c:275:3: return_alloc: Returning allocated memory "newstr".
wget-1.19.5/src/url.c:284:3: return_alloc_fn: Directly returning storage allocated by "url_escape_1".
wget-1.19.5/src/warc.c:1376: var_assign: Assigning: "redirect_location" = storage returned from "url_escape(redirect_location)".
wget-1.19.5/src/warc.c:1381: noescape: Resource "redirect_location" is not freed or pointed-to in "fprintf".
wget-1.19.5/src/warc.c:1387: leaked_storage: Returning without freeing "redirect_location" leaks the storage that it points to.
\# 1385| fflush (warc_current_cdx_file);
\# 1386|
\# 1387|-> return true;
\# 1388| }
\# 1389|
url_escape() really returns a newly allocated memory and it leaks when the warc_write_cdx_record() returns. The memory returned from url_escape() is usually stored in a temporary variable in other parts of the project and then freed. I took the same approach.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
In warc_write_start_record() function, the reutrn value of dup() is
directly used in gzdopen() call and not stored anywhere. However the
zlib documentation says that "The duplicated descriptor should be saved
to avoid a leak, since gzdopen does not close fd if it fails." [1].
This change stores the FD in a variable and closes it in case gzopen()
fails.
[1] https://www.zlib.net/manual.html
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/warc.c:217: open_fn: Returning handle opened by "dup".
wget-1.19.5/src/warc.c:217: leaked_handle: Failing to save or close handle opened by "dup(fileno(warc_current_file))" leaks it.
\# 215|
\# 216| /* Start a new GZIP stream. */
\# 217|-> warc_current_gzfile = gzdopen (dup (fileno (warc_current_file)), "wb9");
\# 218| warc_current_gzfile_uncompressed_size = 0;
\# 219|
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/utils.c:914: open_fn: Returning handle opened by "open". [Note: The source code implementation of the function has been overridden by a user model.]
wget-1.19.5/src/utils.c:914: var_assign: Assigning: "fd" = handle returned from "open(fname, flags, mode)".
wget-1.19.5/src/utils.c:921: noescape: Resource "fd" is not freed or pointed-to in "fstat". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/utils.c:924: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
\# 922| {
\# 923| logprintf (LOG_NOTQUIET, _("Failed to stat file %s, error: %s\n"), fname, strerror(errno));
\# 924|-> return -1;
\# 925| }
\# 926| #if !(defined(WINDOWS) || defined(__VMS))
This seems to be a real issue, since the opened file descriptor in "fd"
would leak. There is also additional check below the "fstat" call, which
closes the opened "fd".
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:4486: alloc_fn: Storage is returned from allocation function "url_string".
wget-1.19.5/src/url.c:2248:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "result" = "xmalloc(size)".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "p" = "result".
wget-1.19.5/src/url.c:2250:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2253:7: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2257:11: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2264:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2270:7: identity_transfer: Passing "p" as argument 1 to function "number_to_string", which returns an offset off that argument.
wget-1.19.5/src/utils.c:1776:11: var_assign_parm: Assigning: "p" = "buffer".
wget-1.19.5/src/utils.c:1847:3: return_var: Returning "p", which is a copy of a parameter.
wget-1.19.5/src/url.c:2270:7: noescape: Resource "p" is not freed or pointed-to in function "number_to_string".
wget-1.19.5/src/utils.c:1774:25: noescape: "number_to_string(char *, wgint)" does not free or save its parameter "buffer".
wget-1.19.5/src/url.c:2270:7: var_assign: Assigning: "p" = "number_to_string(p, url->port)".
wget-1.19.5/src/url.c:2273:3: noescape: Resource "p" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:2287:3: return_alloc: Returning allocated memory "result".
wget-1.19.5/src/http.c:4486: var_assign: Assigning: "hurl" = storage returned from "url_string(u, URL_AUTH_HIDE_PASSWD)".
wget-1.19.5/src/http.c:4487: noescape: Resource "hurl" is not freed or pointed-to in "logprintf".
wget-1.19.5/src/http.c:4513: leaked_storage: Variable "hurl" going out of scope leaks the storage it points to.
\# 4511| {
\# 4512| printwhat (count, opt.ntry);
\# 4513|-> continue;
\# 4514| }
\# 4515| else
There are two conditional branches, which call continue, without freeing memory potentially allocated and pointed to by"hurl" pointer. In fase "!opt.verbose" is True and some of the appropriate conditions in the following if/else if construction, in which "continue" is called, are also true, then the memory allocated to "hurl" will leak.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2434: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/http.c:2434: var_assign: Assigning: "auth_stat" = storage returned from "xmalloc(4UL)".
wget-1.19.5/src/http.c:2446: noescape: Resource "auth_stat" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:70: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "auth_err".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "auth_stat" going out of scope leaks the storage it points to.
\# 2474| /* Creating the Authorization header went wrong */
\# 2475| }
\# 2476|-> }
\# 2477| else
\# 2478| {
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2431: alloc_fn: Storage is returned from allocation function "url_full_path".
wget-1.19.5/src/url.c:1105:19: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:1105:19: var_assign: Assigning: "full_path" = "xmalloc(length + 1)".
wget-1.19.5/src/url.c:1107:3: noescape: Resource "full_path" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:1110:3: return_alloc: Returning allocated memory "full_path".
wget-1.19.5/src/http.c:2431: var_assign: Assigning: "pth" = storage returned from "url_full_path(u)".
wget-1.19.5/src/http.c:2446: noescape: Resource "pth" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:40: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "path".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "pth" going out of scope leaks the storage it points to.
\# 2474| /* Creating the Authorization header went wrong */
\# 2475| }
\# 2476|-> }
\# 2477| else
\# 2478| {
Both "pth" and "auth_stat" are allocated in "check_auth()" function. These are used for creating the HTTP Authorization Request header via "create_authorization_line()" function. In case the creation went OK (auth_err == RETROK), then the memory previously allocated to "pth" and "auth_stat" is freed. However if the creation failed, then the memory is never freed and it leaks.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/ftp.c:1493: alloc_fn: Storage is returned from allocation function "fopen".
wget-1.19.5/src/ftp.c:1493: var_assign: Assigning: "fp" = storage returned from "fopen(con->target, "wb")".
wget-1.19.5/src/ftp.c:1811: leaked_storage: Variable "fp" going out of scope leaks the storage it points to.
\# 1809| if (fp && !output_stream)
\# 1810| fclose (fp);
\# 1811|-> return err;
\# 1812| }
\# 1813|
It can happen, that "if (!output_stream || con->cmd & DO_LIST)" on line #1398 can be true, even though "output_stream != NULL". In this case a new file is opened to "fp". Later it may happen in the FTPS branch, that some error will occure and code will jump to label "exit_error". In "exit_error", the "fp" is closed only if "output_stream == NULL". However this may not be true as described earlier and "fp" leaks.
On line #1588, there is the following conditional free of "fp":
/* Close the local file. */
if (!output_stream || con->cmd & DO_LIST)
fclose (fp);
Therefore the conditional at the end of the function after "exit_error" label should be modified to:
if (fp && (!output_stream || con->cmd & DO_LIST))
fclose (fp);
This will ensure that "fp" does not leak in any case it sould be opened.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
* src/http.c (resp_new): Replace \r\n by space in continuation lines
Fixes#53763
"Malicious website can write arbitrary cookie entries to cookie jar"
HTTP header parsing left the \r\n from continuation line intact.
The Set-Cookie code didn't check and could be tricked to write
\r\n into the cookie jar, allowing a server to generate cookies at will.
* src/openssl.c (init_prng): keep gathering entropy even though we
already have enough
(ssl_connect_with_timeout_callback): reseed PRNG again just before
the handshake
Reported-by: Jeffrey Walton <noloader@gmail.com>
This commit hardens SSL/TLS a bit more in the following ways:
* Explicitly exclude NULL authentication and the 'MEDIUM' cipher list
category. Ciphers in the 'HIGH' level are only considered - this
includes all symmetric ciphers with key lengths larger than 128 bits,
and some ('modern') 128-bit ciphers, such as AES in GCM mode.
* Allow RSA key exchange by default, but exclude it when
Perfect Forward Secrecy is desired (with --secure-protocol=PFS).
* Introduce new option --ciphers to set the cipher list that the SSL/TLS
engine will favor. This string is fed directly to the underlying TLS
library (GnuTLS or OpenSSL) without further processing, and hence its
format and syntax are directly dependent on the specific library.
Reported-by: Jeffrey Walton <noloader@gmail.com>
* src/css-url.c (get_uri_string): Check input length
* fuzz/wget_css_fuzzer.repro/buffer-overflow-6600180399865856:
Add reproducer corpus
Fixes OSS-Fuzz issue #8033.
This is a long standing bug affecting all versions <= 1.19.4.
* src/css-url.c (get_urls_css): Check input string length
* fuzz/wget_css_fuzzer.repro/negative-size-param-5724866467594240:
Add reproducer corpus
Fixes OSS-Fuzz issue #8032.
This is a long standing bug affecting all versions <= 1.19.4.
* src/css-tokens.h: Add enums and fixate values
* src/css.l: Include config.h,
ignore several compiler warnings,
update the grammar to CSS 2.2
Fixes OSS-Fuzz issue #8010 (slowness issue).
This is a long standing bug affecting all versions <= 1.19.4.
Some crafted CSS input was extremely slow / CPU wasting, so it could
be used as a DOS attack against website scanning.
The code/grammar changes were backported from Wget2.x.
* fuzz/Makefile.am: Add wget_ftpls_fuzzer
* fuzz/wget_ftpls_fuzzer.c: New fuzzer
* fuzz/wget_ftpls_fuzzer.dict: Fuzzer dictionary
* fuzz/wget_ftpls_fuzzer.in/starter: Starting corpus
* src/ftp-ls.c: Parsing function take FILE * as argument,
new function ftp_parse_ls_fp()
* src/ftp.c: Remove static from freefileinfo()
* src/ftp.h: Add ftp_parse_ls_fp() and freefileinfo()
* fuzz/Makefile.am: Add wget_html_fuzzer
* fuzz/wget_html_fuzzer.c: New fuzzer
* fuzz/wget_html_fuzzer.dict: HTML dictionary for fuzzing
* fuzz/wget_html_fuzzer.in: Initial corpora
* src/html-url.c: Add new function get_urls_html_fm()
* src/html-url.h: Add ne function get_urls_html_fm()
* src/wget.h: Fix define for fopen_wgetrc()
* fuzz/wget_options_fuzzer.c: Add fopen_wget() and fopen_wgetrc()
* src/utils.c: Use fopen_wgetrc() for config files,
don't read from stdin when fuzzing
* src/wget.h: Define fopen as fopen_wget when fuzzing,
define fopen_wgetrc as fopen when not fuzzing
* src/ftp.c (ftp_loop_internal): Set warc_tmp to NULL after ffclose()
* src/init.c (cleanup): Set output_stream to NULL after fclose()
* src/log.c (log_close): Set global stream vars to NULL after closing
* src/recur.c (retrieve_tree): Set rejectedlog to NULL after closing
* src/warc.c (warc_close): Set stream vars to NULL after closing
* Makefile.am: Add fuzz/ to SUBDIRS
* cfg.mk: Fix 'make syntax-check'
* configure.ac: Add --enable-fuzzing
* fuzz/Makefile.am: New file
* fuzz/README.md: New file
* fuzz/fuzzer.h: New file
* fuzz/get_all_corpora: New file
* fuzz/get_ossfuzz_corpora: New file
* fuzz/glob_crash.c: New file
* fuzz/main.c: New file
* fuzz/run-afl.sh: New file
* fuzz/run-clang.sh: New file
* fuzz/view-coverage.sh: New file
* fuzz/wget_options_fuzzer.c: New file
* fuzz/wget_options_fuzzer.dict: New file
* src/init.c (cleanup): Free more resources
* src/main.c (init_switches): Initialize only once,
(print_usage): Don't print if TESTING is defined
* src/utils.h: Include wget.h
