Commit Graph

3992 Commits

Author SHA1 Message Date
Tim Rühsen
1098de2b49 Do not hard-code -ldl in fuzz/Makefile.am
* configure.ac: Search for dlopen and add library to $FUZZ_LIBS
* fuzz/Makefile.am: Link with $FUZZ_LIBS instead of -ldl

This fixes linking on BSD systems.

Reported-by: Nam Nguyen
2019-02-19 15:11:04 +01:00
Tim Rühsen
c7f31d62b2 * doc/wget.texi: Correct --logfile -> --output-file 2019-02-14 10:39:54 +01:00
Darshit Shah
c53552e8cd Update gnulib 2019-02-10 11:51:03 +01:00
Darshit Shah
8b975e83d3 Update copyright statements 2019-02-10 11:50:17 +01:00
Leon Klingele
9e7c1554bd docs: --no-cache also sets the 'Cache-Control: no-cache' header
* doc/wget.texi: Add Cache-Control to docs
* src/wget.h: Add Cache-Control to comment of SEND_NOCACHE

Copyright-paperwork-exempt: Yes
2019-01-23 22:04:01 +01:00
Tim Rühsen
caf30fbe28 * .gitlab-ci.yml: Add minimal build 2019-01-20 19:58:10 +01:00
Tim Rühsen
f1eda916a6 * src/init.c (cleanup): Check HAVE_HSTS
Reported-by: Simon Dales
2019-01-20 19:42:53 +01:00
André Wolski
5dceb6626b NTLM restart authentication (trivial change)
* src/http-ntlm.c (ntlm_input): Continue on NTLMSTATE_LAST,
  error on NTLMSTATE_TYPE3

The code comes from the cURL project. Thanks to Daniel Stenberg
for donating the code.

https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00030.html
2018-12-31 10:24:27 +01:00
Tim Rühsen
fc65b4521a * contrib/spell-checker: Remove trailing whitespace 2018-12-28 19:05:42 +01:00
Tim Rühsen
fdb7fe613c Fix typos detected by codespell (via contrib/spell-checker) 2018-12-28 18:58:15 +01:00
Tim Rühsen
c0427ebf3c * contrib/spell-checker: Add script for spell checking 2018-12-28 18:55:20 +01:00
Tim Rühsen
af16b0f38f * fuzz/Makefile.am: Fix order of libraries for linking 2018-12-27 20:48:36 +01:00
Tim Rühsen
6ba1cb587c * src/wget.h: #undef _Noreturn when building with C++ 2018-12-27 20:47:58 +01:00
Tim Rühsen
5811c2222b * src/gnutls.c (ssl_connect_wget): Fix call to gnutls_set_default_priority() 2018-12-27 20:46:55 +01:00
Tim Rühsen
70cddf3e29 * NEWS: Add release changes for 1.20.1 2018-12-26 20:35:41 +01:00
Tim Rühsen
3cdfb594cf Don't save user/pw with --xattr
Also the Referer info is reduced to scheme+host+port.

* src/ftp.c (getftp): Change params of set_file_metadata()
* src/http.c (gethttp): Change params of set_file_metadata()
* src/xattr.c (set_file_metadata): Remove user/password from origin URL,
  reduce Referer value to scheme/host/port.
* src/xattr.h: Change prototype of set_file_metadata()
2018-12-26 14:38:24 +01:00
Tim Rühsen
c125d24762 Don't use extended attributes (--xattr) by default
* src/init.c (defaults): Set enable_xattr to false by default
* src/main.c (print_help): Reverse option logic of --xattr
* doc/wget.texi: Add description for --xattr

Users may not be aware that the origin URL and Referer are saved
including credentials, and possibly access tokens within
the urls.
2018-12-26 14:06:38 +01:00
Tim Rühsen
7c0a30d724 * .travis.yml: Email to wget-dev instead bug-wget mailing list 2018-12-13 15:22:17 +01:00
Darshit Shah
3becef17cb * NEWS: Prepare for new version 2018-11-30 01:11:19 +01:00
Darshit Shah
8f780fa274 * contrib/make-release: Add a small checklist for pending tasks 2018-11-13 17:44:51 +01:00
Darshit Shah
e55916e5de Prepare NEWS for new release 2018-11-13 16:24:17 +01:00
Darshit Shah
c4a6b89b2e * configure.ac: gnulib now expects autoconf >=2.63 2018-11-13 16:20:19 +01:00
Darshit Shah
3d4e1f47a1 * gnulib: Update library 2018-11-13 16:20:19 +01:00
Jay Satiro
61271d87f6 * src/init.c: Stop freeing the pointer returned by ws_mypath()
.. since ws_mypath() saves the address it returns in a static pointer
for reuse, to also be returned in later calls.
2018-11-13 15:51:51 +01:00
Darshit Shah
2bc2d2f803 * src/ftp.c(ftp_retrieve_glob): Honor {accept,reject}-regex switches as well 2018-11-13 15:51:51 +01:00
Darshit Shah
8c741da256 * src/ftp.c (ftp_retrieve_glob): Refactor to prevent looping over listing multiple times 2018-11-13 15:51:51 +01:00
Tim Rühsen
d3383d619b * .gitlab-ci.yml: Split into GnuTLS and OpenSSL build 2018-11-11 18:43:46 +01:00
Tim Rühsen
62dced2f43 * Makefile.am: dist clean po/stamp-po 2018-11-11 17:51:37 +01:00
Tim Rühsen
04116409a8 Remove auto-generated files from po/ 2018-11-11 16:58:09 +01:00
Tim Rühsen
7b8325b416 Add VPATH build 2018-11-11 16:57:41 +01:00
Tim Rühsen
11fad3fa72 Revert "Bail out on unexpected 416 server errors"
This reverts commit 6f3b995993.

The code is obviously wrong, see https://savannah.gnu.org/bugs/?54963
Also, the example from the original post doesn't work any more.
With other words, the broken server behavior has been fixed meanwhile.
2018-11-09 16:16:43 +01:00
Rosen Penev
a3643c6076 openssl: Do not use engines when OpenSSL does not support
* src/openssl.c: Check for OPENSSL_NO_ENGINE before
 including openssl/engine.h and before calling ENGINE_load_builtin_engines()

Fixes compilation with no engines compiled.

Copyright-paperwork-exempt: Yes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-11-09 16:01:51 +01:00
Tim Rühsen
ed23e682da Fix HTTPS Perl tests
* tests/SSLTest.pm: Rename server cert and key file
* tests/Test-https*.px: Fix and remove OpenSSL hard-coding
* tests/certs/create-certs.sh: Script to generate test files
* tests/certs/*-template.txt: GnuTLS template files for certs and crl
* tests/certs/*.pem: Keys, certs, crls
* tests/certs/README: Removed commands, link to create-certs.sh
2018-11-09 15:47:53 +01:00
Kapus, Timotej
6d7cd9313c Replace some loops with string.h functions
* src/init.c: Replace loop with strspn
* src/url.c: Replace loop with strrchr

Copyright-paperwork-exempt: Yes
2018-10-28 10:36:46 +01:00
Luiz Angelo Daros de Luca
7903dbc9d8 * .gitmodules: Use https:// instead of git:// for gnulib
git:// does not work over http proxy

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:55:44 +02:00
Luiz Angelo Daros de Luca
fd85ac9cc6 * src/host.c (sufmatch): Fix dot-prefixed domain matching
Current sufmatch does not match when domain is dot-prefixed.
The example of no_proxy in man (.mit.edu) does use a dot-prefixed
domain.

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:54:26 +02:00
Tim Rühsen
21daa24e72 * src/convert.c (convert_links): Fix fallthrough 2018-10-26 22:52:41 +02:00
Darshit Shah
6d6679ba86 * bootstrap: Update script from gnulib 2018-10-22 16:45:58 +02:00
Darshit Shah
d18f78b86b * gnulib: Update library 2018-10-22 16:45:58 +02:00
Tim Rühsen
85310bfa58 * .lgtm.yml: New file to add LGTM to Gitlab.com CI 2018-10-19 12:32:01 +02:00
Tim Rühsen
b29854528a * configure.ac: Fix build issue with libgpgme 2018-10-16 10:31:54 +02:00
Tim Rühsen
3d518f125c * fuzz/*_fuzzer.in/*: Update fuzzer corpora 2018-10-14 15:15:49 +02:00
Nikos Mavrogiannopoulos
c11cc83d9e Enable post-handshake auth under gnutls on TLS1.3 2018-10-08 15:55:48 +02:00
Tim Rühsen
0727b8f3a9 * src/http.c (resp_new): Fix code to avoid false positive by clang 2018-09-20 14:59:06 +02:00
Tim Rühsen
88a49c1e41 * src/convert.c (convert_links): Fix code to avoid false positive by clang 2018-09-20 14:58:27 +02:00
Tim Rühsen
02afe1e41c Add support for PCRE2 pattern matching
* configure.ac: Check for libpcre2-8
* src/init.c (choices): Test for HAVE_LIBPCRE2
* src/main.c (main): Set regex compile and match functions
* src/options.h: Test for HAVE_LIBPCRE2
* src/utils.c: Include pcre2.h, add functions
  compile_pcre2_regex() and match_pcre2_regex()
* src/utils.h: Declare compile_pcre2_regex() and match_pcre2_regex()

Fixes #54677
Reported-by: Noël Köthe
2018-09-19 16:22:25 +02:00
Tim Rühsen
79be99aff4 Add . to perl path for all perl tests
* tests/*.px: Add -I . to the shebang

This allows perl test to be run from tests/ directory, e.g. via
  ./Test--post-file.px
2018-09-07 10:48:14 +02:00
Tomas Hozza
2bbdfd76da Add TLS 1.3 support for GnuTLS
* doc/wget.texi: Add "TLSv1_3" to --secure-protocol
* src/gnutls.c (set_prio_default): Use GNUTLS_TLS1_3 where needed

Wget currently allows specifying "TLSv1_3" as the parameter for
--secure-protocol option. However it is only implemented for OpenSSL
and in case wget is compiled with GnuTLS, it causes wget to abort with:
GnuTLS: unimplemented 'secure-protocol' option value 6

GnuTLS contains TLS 1.3 implementation since version 3.6.3 [1]. However
currently it must be enabled explicitly in the application of it to be
used. This will change after the draft is finalized. [2] However for
the time being, I enabled it explicitly in case "TLSv1_3" is used with
--secure-protocol.

I also fixed man page to contain "TLSv1_3" in all listings of available
parameters for --secure-protocol

[1] https://lists.gnupg.org/pipermail/gnutls-devel/2018-July/008584.html
[2] https://nikmav.blogspot.com/2018/05/gnutls-and-tls-13.html

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-09-07 09:56:02 +02:00
Tomas Korbar
7ddcebd61e Avoid creating empty wget-log when using -O and -q in background
* src/log.c (check_redirect_output): Check for quiet mode
2018-08-29 12:34:03 +02:00
Tomas Hozza
2f451dbf4e * src/warc.c (warc_write_cdx_record): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772): - REAL ERROR
wget-1.19.5/src/warc.c:1376: alloc_fn: Storage is returned from allocation function "url_escape".
wget-1.19.5/src/url.c:284:3: alloc_fn: Storage is returned from allocation function "url_escape_1".
wget-1.19.5/src/url.c:255:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:255:3: var_assign: Assigning: "newstr" = "xmalloc(newlen + 1)".
wget-1.19.5/src/url.c:258:3: var_assign: Assigning: "p2" = "newstr".
wget-1.19.5/src/url.c:275:3: return_alloc: Returning allocated memory "newstr".
wget-1.19.5/src/url.c:284:3: return_alloc_fn: Directly returning storage allocated by "url_escape_1".
wget-1.19.5/src/warc.c:1376: var_assign: Assigning: "redirect_location" = storage returned from "url_escape(redirect_location)".
wget-1.19.5/src/warc.c:1381: noescape: Resource "redirect_location" is not freed or pointed-to in "fprintf".
wget-1.19.5/src/warc.c:1387: leaked_storage: Returning without freeing "redirect_location" leaks the storage that it points to.
\# 1385|     fflush (warc_current_cdx_file);
\# 1386|
\# 1387|->   return true;
\# 1388|   }
\# 1389|

url_escape() really returns a newly allocated memory and it leaks when the warc_write_cdx_record() returns. The memory returned from url_escape() is usually stored in a temporary variable in other parts of the project and then freed. I took the same approach.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:25:34 +02:00