Commit Graph

4419 Commits

Author SHA1 Message Date
Tim Rühsen
b32cc7afe0 Fix HSTS matching
* src/hsts.c (hsts_find_entry): Check for includeSubdomains,
  (test_hsts_new_entry): Fix test,
  (test_hsts_url_rewrite_superdomain): Improve test.

Reported-by: Hanno Böck <hanno@hboeck.de
2024-02-24 19:12:59 +01:00
Darshit Shah
3aa53a6220 Delete some redundant tests
* tests/Makefile.am: Remove some tests that are redundant with the
  Python testenv
* tests/Test-auth-basic.px: Delete file
* tests/Test-auth-no-challenge.px: Same
* tests/Test-auth-no-challenge-url.px: Same
* tests/Test-auth-retcode.px: Same
* tests/Test-auth-with-content-disposition.px: Same
* tests/Test-k.px: Same
2024-02-19 19:23:25 +05:30
Darshit Shah
0e0cdc2409 * Makefile.am: Ignore some lcov errors, allowing the tests to run through 2024-02-19 19:21:03 +05:30
Darshit Shah
84a75ace88 * README: Add a link to the COPYING file to meet the GNU Coding Standards 2024-02-19 18:43:12 +05:30
Darshit Shah
35204ab5d7 * bootstrap: Update script from gnulib 2024-02-19 18:37:51 +05:30
Darshit Shah
e377b80863 * gnulib: Update gnulib 2024-02-19 18:35:26 +05:30
Darshit Shah
f973f4857a * Update copyright year to 2024 2024-02-19 18:33:43 +05:30
Darshit Shah
f4da5456c2 Add tests for --convert-links option
* testenv/Makefile.am: Add two new tests, Test-k.py and Test-https-k.py
* testenv/Test-k.py: New file. Add a test based on tests/Test-k.px
* testenv/Test-https-k.py: New file. Add a new test to ensure that the
  protocol of the original host URL is retained when creatign absolute
  links.

This test is added as a result of an issue reported on StackExchange:
https://superuser.com/questions/1348940/making-wgets-convert-links-respect-http-vs-https
2024-02-19 18:27:38 +05:30
Darshit Shah
dfb9f65fc3 * testenv/conf/expected_files.py: Aesthetic changes only. Format file with black 2024-02-19 18:27:32 +05:30
Darshit Shah
642e740182 * testenv/conf/expected_files.py: Nicer diff printing on error 2024-02-19 18:27:19 +05:30
Darshit Shah
28009a048f * SECURITY.md: Add a file stating how to report security issues 2024-02-18 17:07:19 +05:30
Tim Rühsen
4100339a2b Parse 'srcset' HTML attr for 'source' HTML tag.
* src/html-url.c (struct known_tag): Use tag_handle_img() for 'source' tag.
* testenv/Test-recursive-include.py: Extend test.
2024-01-27 19:50:13 +01:00
Tim Rühsen
bedeb7dc27 * src/netrc.c (parse_netrc_fp): Add fetchmail compatibility (user and passwd)
Reported-by: Gerald Pfeifer <gerald@pfeifer.com>
2023-11-26 19:06:11 +01:00
Tim Rühsen
25525f8037 Fix confusing 'Cannot write to ... (Success)' message
* src/http.c (gethttp): Store/restore errno value.

Reported-by: Christian Rosentreter, Andries E. Brouwer
2023-10-22 14:01:09 +02:00
Jan-Michael Brummer
8e8900613c Add libproxy support
Add support for libproxy, which is capable to extract desktop
environment proxy configurations from dozens of systems and platforms.
This also enables wget to handle pac/wpad proxy server.

* configure.ac: Add check for libbproxy.
* src/retr.c (getproxy): Retrieve proxy via libproxy.

Copyright-paperwork-exempt: Yes
2023-10-20 18:08:01 +02:00
Tim Rühsen
73d24b2779 * src/retr.c: Fix sc_prohibit_empty_lines_at_EOF 2023-08-03 11:32:09 +02:00
Tim Rühsen
04ab356669 Fix crash when printing download rate
If the download rate is TB/s, a read buffer overflow happended
that either caused a crash or printed whatever string was pointed to.

* src/retr.c (retr_rate): Add missing array entrie for TB/s and Tb/s,
  (test_retr_rate): New test function.
* tests/unit-tests.c (all_tests): Run test 'test_retr_rate'.
* tests/unit-tests.h: Add prototype for test_retr_rate.

Reported-by: Wiebe Cazemier <wiebe@halfgaar.net>
2023-08-03 11:19:41 +02:00
Christian Weisgerber
4d99bb1ff1 * tests/Makefile.am: Remove './' from for portability (OpenBSD)
Copyright-paperwork-exempt: Yes
2023-08-03 10:39:56 +02:00
Yaakov Selkowitz
9c8668048d testenv: fix for Python 3.12
* testenv/server/http/http_server.py (HTTPSServer): Update for
  ssl.SSLContext APIs instead of deprecated ssl.wrap_socket().

ssl.wrap_socket() was deprecated in 3.7 and removed in 3.12.
This should be compatible back to 3.6 (RHEL 8 and newer).

Copyright-paperwork-exempt: Yes
2023-07-16 14:08:30 +02:00
Tim Rühsen
3583fa0c61 * src/url.c (test_uri_merge): Fix check 2023-07-01 18:20:32 +02:00
Tim Rühsen
834d090bf6 Add new unit test test_uri_merge()
* src/url.c: New test function test_uri_merge().
* tests/unit-tests.c (tests/unit-tests.c): Call test_uri_merge().
* tests/unit-tests.h (tests/unit-tests.h): Declare test_uri_merge().
2023-07-01 18:14:00 +02:00
Darshit Shah
fbbdf9ea01 Ensure that spaces are quoted when converting links
* src/convert.c(convert_links): Print the actual quoted newname when printing DEBUG output
  (local_quote_string): Also quote the ' ' charcter as %20. While it is okay
  to leave the characted as-is, quoting it covers more edge cases.
  And it should resolve a >10 year old bug with CSS url() parameters not being quoted

Bug-Id: 64082
Reported-By: Ethan Gibbs <ethan@snowsign.net>
Discussed-At: https://stackoverflow.com/q/13300017
2023-05-16 18:46:19 +02:00
Darshit Shah
5409cbcee2 Add new test to ensure CSS url() encoding
url() parameters in CSS cannot have spaces in them. Ensure that Wget does not do that
when using --convert-links

* testenv/test_css_url.py: New file
* testenv/Makefile: Add test_css_url.py to tests

Bug-Id: 64082
2023-05-16 00:11:25 +02:00
Darshit Shah
0fea7bc076 Automatically verify if commit author has assigned copyrights in the past
* contrib/commit-check: Add new script
* .gitlab-ci.yml: Add new test in the CI pipeline
2023-05-14 21:55:01 +02:00
Jan Palus
6ca59f4d60 * testenv/conf/expected_files.py: Ignore common.conf
Copyright-paperwork-exempt: Yes
2023-05-14 21:55:01 +02:00
Darshit Shah
27a832aee9 * AUTHORS: Rework file to prepare for autmated testing 2023-05-14 21:55:01 +02:00
Darshit Shah
719ab50a18 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2023-05-11 03:02:48 +02:00
Darshit Shah
76ab37bff9 contrib/make-release: Update regexes to match the NEWS file 2023-05-11 02:22:45 +02:00
Darshit Shah
774705838f cfg.mk: Replace uses of filesystem with file system 2023-05-11 01:03:23 +02:00
Darshit Shah
e234354080 Update NEWS 2023-05-11 00:48:44 +02:00
Darshit Shah
3a5ca80a1a bootstrap.sh: Update script from gnulib 2023-05-10 23:59:55 +02:00
Darshit Shah
c3fadea0a8 gnulib: Pull forward 2023-05-10 23:57:52 +02:00
jinfuchiang
2b723806a5 Update deprecated option '--html-extension' to '--adjust-extension'
* doc/wget.texi: Update option '--html-extension' to '--adjust-extension'.
  Renamed in Wget 1.12 to better reflect its behavior.

Copyright-paperwork-exempt: Yes
2023-04-09 20:29:07 +02:00
Tim Rühsen
9a35fe609c Don't write core dump if --secure-option value isn't suppported. 2023-03-19 17:39:45 +00:00
Tim Rühsen
d96d20630b * src/retr.c (fd_read_body): Increase bufsize from 8k to 64k 2023-02-18 17:26:23 +01:00
Tim Rühsen
c77c95033a * src/retr.c (fd_read_body): Simplify gzip initialization 2023-02-18 17:26:23 +01:00
Tim Rühsen
af1100f299 * src/retr.c (fd_read_body): Use MAX instead of max 2023-02-18 17:26:23 +01:00
Tim Rühsen
77929eda1b Simplify url_error function.
* src/url.c (url_error): simplify, remove url arg, return const char *.
* src/url.h (url_error): remove url arg, return const char *.
* src/html-url.c (get_urls_file): Simplify call to url_error(),
  remove call to free().
* src/http.c (metalink_from_http): Likewise.
* src/main.c (main): Likewise.
* src/metalink.c (retrieve_from_metalink, fetch_metalink_file): Likewise.
* src/recur.c (retrieve_tree): Likewise.
* src/res.c (res_retrieve_file): Likewise.
* src/retr.c (retrieve_url, retrieve_from_file): Likewise.
2023-02-18 17:25:56 +01:00
Tim Rühsen
218f6fee30 * src/url.c: Refactored url_error() 2023-02-18 17:25:56 +01:00
Tim Rühsen
2339d79b05 * tests/valgrind-suppressions: Add rule for 'strcmp-sse2.S in libdl.so' 2023-02-18 17:25:56 +01:00
Tim Rühsen
dcd2ed739e * configure.ac: Remove CC flags -Wc90-c99-compat and -Wlong-long 2023-02-17 14:00:02 +01:00
Tim Rühsen
24e876357a * fuzz/*: Use grep >/dev/null instead of grep -q 2023-02-17 13:46:58 +01:00
Tim Rühsen
8247a34521 * src/retr.c: Don't include hash.h 2023-02-17 13:46:58 +01:00
Tim Rühsen
4223996930 Update copyright years 2023-02-17 13:46:58 +01:00
Tim Rühsen
27d3fcba33 * gnulib: Update 2023-02-17 13:46:58 +01:00
Tim Rühsen
c69030a904 * configure.ac: Disable nettle if NTLM is explicitly disabled 2022-12-11 13:31:38 +01:00
Tim Rühsen
485217d0ff * configure.ac: Allow disabling NTLM if nettle present (Savannah #63431) 2022-12-10 16:43:38 +01:00
Tim Rühsen
9835085544 * fuzz/Makefile.am: Add -lz to fuzzer libs 2022-09-24 18:41:31 +02:00
Tim Rühsen
aab539bb44 * src/main.c (print_help): Add --retry-on-host-error to help text 2022-03-20 21:07:54 +01:00
Tim Rühsen
cb114fbbf7 Fix HSTS portability by using int64_t instead of time_t.
* src/hsts.c: Use int64_t instead of time_t.
* src/http.c: Use int64_t for parsing Strict-Transport-Security.
2022-03-20 20:57:10 +01:00