Commit Graph

3653 Commits

Author SHA1 Message Date
Tim Rühsen
302aa79abb Fallback to pod2man without utf-8 on error
* doc/Makefile.am: Fallback to pod2man without utf-8 on error
2016-05-27 16:44:50 +02:00
Ander Juaristi
5224d752a5 Correct HSTS debug message
* src/main.c (save_hsts): save the in-memory HSTS database to a file
   only if something changed.
 * src/hsts.c (struct hsts_store): new field 'changed'.
   (hsts_match): update field 'changed' accordingly.
   (hsts_store_entry): update field 'changed' accordingly.
   (hsts_store_has_changed): new function.
 * src/hsts.h (hsts_store_has_changed): new function.
2016-05-26 16:37:51 +02:00
Ander Juaristi
2aaf12990c Check the HSTS file is not world-writable
* hsts.c (hsts_file_access_valid): check that the file is a regular
   file, and that it's not world-writable.
   (hsts_store_open): if the HSTS database file does not meet the
   above requirements, disable HSTS at all.
2016-05-26 16:29:29 +02:00
Tim Rühsen
2e6f629061 Improve description of Perl libraries needed for test suite
* README.checkout: Improve text about Perl libraries
2016-05-24 09:38:15 +02:00
Tim Rühsen
a952f81f3e Remove special handling for Emacs in progress bar code
* src/progress.c: Remove special 'emacs' code

Fixes #47989
2016-05-23 21:46:29 +02:00
Jernej Simončič
42cc84b6b6 Fix xsleep() for Windows (trivial change)
* src/mswindows.c (xsleep): Fix check for number of seconds
2016-04-25 15:50:23 +02:00
Sergio Gelato
96ab9cad88 More accurate log message from do_conversion()
* src/iri.c (do_conversion): More accurate log message
2016-04-17 15:28:48 +02:00
Tim Rühsen
268163444d Include sys/select.h if HAVE_LIBCARES
* src/hosts.c: Include sys/select.h if HAVE_LIBCARES

Reported-by: Gisle Vanem <gvanem@yahoo.no>
2016-04-17 14:18:55 +02:00
Gisle Vanem
53800415a9 Fix Windows gnulib/c-ares incompatibility of select()
* src/host.c: Undef 'select' on Windows
2016-04-17 14:15:51 +02:00
Tim Rühsen
528770ca8b Set X flags for python tests
* testenv/*.py: Set eXecute flags

Regression from commit 926e42d467
2016-04-15 21:36:51 +02:00
Ander Juaristi
2f1c6a05c8 Strictly comply with RFC 6797
* src/hsts.c (hsts_store_entry): strictly comply with RFC 6797.

RFC 6797 states in section 8.1 that the UA's cached information should
only be updated if:

    "either or both of the max-age and includeSubDomains header field
    value tokens are conveying information different than that already
    maintained by the UA."
2016-04-11 16:44:47 +02:00
Ander Juaristi
33d860e1ef Correct HSTS database file description
* src/hsts.c (hsts_store_dump): s/[:port]/<port>/
2016-04-11 16:44:41 +02:00
moparisthebest
99fc712f21 Implement tests for new pinnedpubkey option
* testenv/Makefile.am: Add new tests
* testenv/Test-pinnedpubkey-der-https.py: New test
* testenv/Test-pinnedpubkey-der-no-check-https.py: New Test
* testenv/Test-pinnedpubkey-hash-https.py: New test
* testenv/Test-pinnedpubkey-hash-no-check-fail-https.py: New test
* testenv/Test-pinnedpubkey-pem-fail-https.py: New test
* testenv/Test-pinnedpubkey-pem-https.py: New test
* testenv/certs/README: How to generate public keys with openssl tool
* testenv/certs/server-pubkey.der: New key file (DER format)
* testenv/certs/server-pubkey.pem: New key file (PEM format)
2016-04-11 16:25:09 +02:00
moparisthebest
54746578e9 Implement --pinnedpubkey option to pin public keys
* doc/wget.texi: Add description for --pinnedpubkey
* src/gnutls.c: New function pkp_pin_peer_pubkey(),
  (ssl_check_certificate): Check pinned cert via pkp_pin_peer_pubkey()
* src/init.c: Add option --pinnedpubkey
* src/main.c: Add option --pinnedpubkey
* src/openssl.c: New function pkp_pin_peer_pubkey(),
  (ssl_check_certificate): Check pinned cert via pkp_pin_peer_pubkey()
* src/options.h: Add new option variable 'pinnedpubkey'
* src/utils.c: New functions wg_pubkey_pem_to_der(), wg_pin_peer_pubkey()
* src/utils.h: Add prototype for wg_pin_peer_pubkey()
2016-04-11 16:18:05 +02:00
Tim Rühsen
926e42d467 Use test file name for temp working directory name
* testenv/test/base_test.py (__init__): Use test file name for name,
  remove 'name' parameter
* testenv/test/http_test.py (__init__): Remove 'name' parameter
* testenv/*.py: Remove TEST_NAME

Using a fixed string (TEST_NAME) to build the working directory
for testing caused random failures (or successes) when tests
share the same TEST_NAME value. Not easy to find without digging
into the python test suite code.
We now use the test file name, which is unique within the test
environment.
2016-04-11 15:35:05 +02:00
Tim Rühsen
d286d5408b Fix testenv/Test--rejected-log.py
* testenv/Test--rejected-log.py: Add missing tabs in expected output
2016-04-11 15:24:04 +02:00
Tim Rühsen
9883f9fb10 Fail python tests when post_hook errors
* testenv/test/base_test.py (__exit__): Return self.tests_passed
  (__test_cleanup): Set self.tests_passed to False on exception
2016-04-11 15:23:38 +02:00
Darshit Shah
d26377053d Print the fingerprint instead of the raw pointer in debugging message
* src/metalink.c (retrieve_from_metalink): Fix debug message to print the
fingerprint instead of a pointer.

* Do not delete the ChangeLog file since it is required by the Makefile
and breaks compilation
2016-03-29 15:09:04 +02:00
Darshit Shah
722675553c Revert "Print the fingerprint instead of the raw pointer in debugging message"
This reverts commit b916595168.
2016-03-29 15:07:29 +02:00
Tim Rühsen
281ad7dfb9 Fixed URLs and references in wget.texi
* wget.texi: Replace server.com by example.com,
  replace ftp://wuarchive.wustl.edu by https://example.com,
  use HTTPS instead of HTTP where possible,
  fix list archive reference,
  remove reference to wget-notify@addictivecode.org,
  change bugtracker URL to bugtracker on Savannah,
  replace yoyodyne.com by example.com,
  fix URL to VMS port
2016-03-28 18:33:33 +02:00
Giuseppe Scrivano
f3e63f0071 * metalink.c (retrieve_from_metalink): Fix typo 2016-03-25 16:46:39 +01:00
Giuseppe Scrivano
b916595168 Print the fingerprint instead of the raw pointer in debugging message
* src/metalink.c (retrieve_from_metalink): Fix debug message to print the
fingerprint instead of a pointer.
2016-03-25 16:23:19 +01:00
Tim Rühsen
76ef65b23c Add options --bind-dns-address and --dns-servers
* README.checkout: Add description for libares
* configure.ac: Add check for libares
* doc/wget.texi: Add docs for the new options
* src/build_info.c.in: Add +/-cares for --version output
* src/host.c:
  (merge_address_lists): New static function
  (address_list_from_hostent): New static function
  (wait_ares): New static function
  (callback): New static function
  (lookup_host): Add libares resolver code
* src/init.c: Add new options,
  (cleanup): Add cleanup code
* src/main.c: Add global libares channel variable
  (cmdline_option option_data): Add new options
  (print_help): Add short descriptions
  (main): Add libares init code
* src/options.h (struct options): Add option members

The new options allow to specify alternative DNS servers and
an alternate packet route for the resolver packets.
Wget has to built with libares, enabled at configure time by
./configure --with-cares.
2016-03-23 09:26:22 +01:00
Tim Rühsen
d7726f8a13 Fix SNI server names with trailing dot(s)
* src/gnutls.c (ssl_connect_wget, ssl_check_certificate): Fix SNI server name
* src/openssl.c (ssl_connect_wget, ssl_check_certificate): Fix SNI server name

Fixes #47408
2016-03-16 11:23:51 +01:00
Tim Rühsen
598445ebd1 Fix links to original Robots Exclusion Standard
* doc/wget.texi: Fix links
2016-03-10 16:20:05 +01:00
Darshit Shah
7cb9efa668 Fix assertion in Progress bar
* src/progress.c (create_image): Fix off-by-one error in assert()
    statement for progress bar width.
    Reported-By: Gisle Vanem <gvanem@yahoo.no>
2016-03-05 13:27:46 +01:00
Giuseppe Scrivano
44aedd8321 src/url.c: fix make syntax-check 2016-03-03 09:40:39 +01:00
Maks Orlovich
c28f51aadf Parse <img srcset> attributes, they have image URLs.
* src/convert.h: Add link_noquote_html_p to permit rewriting URLs deep
                 inside attributes without adding extraneous quoting
* src/convert.c (convert_links): Honor link_noquote_html_p
* src/html_url.c (tag_handle_img): New function. Add srcset parsing.
2016-03-03 09:38:45 +01:00
Darshit Shah
7099f48998 Sanitize value sent to memset to prevent SEGFAULT 2016-03-01 08:11:13 +01:00
Darshit Shah
75e5be7aad Update documentation about bahviour of -c
* docs/wget.texi: -c will restart download from scratch if server
    does not support RANGE.

    Reported-By: David Chavez
    http://stackoverflow.com/questions/30147332/unexpected-behavior-of-wget
2016-03-01 08:10:59 +01:00
Tim Rühsen
100da11312 Fix writing WARC-Target-URI value
src/warc.c: Add function warc_write_header_uri(),
            Use it for creating WARC-Target-URI

Fixes #47281
2016-02-27 23:08:28 +01:00
Darshit Shah
cceb5932b4 Remove pointer to unused Mailing List
* MAILING-LISTS: Remove pointer to old, unused mailing list for bug
    reports
2016-02-13 01:39:03 +01:00
Tim Rühsen
cacac6f996 Retain value of errno in logprintf(), logputs() even better
* src/log.c (logprintf,logputs): Save&Restore value of errno

Reported-by: Gisle Vanem <gvanem@yahoo.no>
2016-02-11 10:53:02 +01:00
Tim Rühsen
3056617e9c Retain value of errno in logprintf()
* src/log.c (logprintf): Save&Restore value of errno

Reported-by: Gisle Vanem <gvanem@yahoo.no>
2016-02-10 15:28:10 +01:00
Tim Rühsen
c190677cab Set AM_SILENT_RULES to yes by default
* configure.ac: Set AM_SILENT_RULES to yes by default
2016-02-01 21:10:33 +01:00
Ander Juaristi
4c2dc5010f Enforce 'RejectHeader' rule in tests
* server/http/http_server.py (_Handler.RejectHeader): enforce
   'RejectHeader' rule.
2016-02-01 20:48:00 +01:00
Tim Rühsen
b30500f0f4 Fix Test-iri-forced-remote
* tests/Test-iri-forced-remote.px: Fix encodings
2015-12-20 21:32:06 +01:00
Eli Zaretskii
59b920874d Support non-ASCII URLs
* src/url.c [HAVE_ICONV]: Include iconv.h and langinfo.h.
(convert_fname): New function.
[HAVE_ICONV]: Convert file name from remote encoding to local
encoding.
(url_file_name): Call convert_fname.
(filechr_table): Don't consider bytes in 128..159 as control
characters.

* tests/Test-ftp-iri.px: Fix the expected file name to match the
new file-name recoding.  State the remote encoding explicitly on
the Wget command line.

* NEWS: Mention the URI recoding when built with libiconv.
2015-12-18 20:54:39 +01:00
Giuseppe Scrivano
9a6e63bee9 * NEWS: Prepare new development cycle 2015-12-18 11:29:59 +01:00
Tim Rühsen
cbbeca2af4 Cleanup code
* src/iri.c (do_conversion): Code cleanup
2015-12-17 21:01:50 +01:00
Eli Zaretskii
93c1517c40 Set URI encoding when redirected
* src/retr.c (retrieve_url): Set URI on redirection
2015-12-17 15:27:43 +01:00
Tim Rühsen
bf5d7e9236 Remove requesting X/Open 5, POSIX 1995
* src/sysdep.h: Remove #define _XOPEN_SOURCE 500
2015-12-17 12:11:53 +01:00
Eli Zaretskii
94d9b68db9 Avoid hanging on MS-Windows when invoked with --connect-timeout
* src/connect.c (connect_to_ip) [WIN32]: Don't call fd_close if
the connection timed out, to avoid hanging.
2015-12-16 15:06:45 +01:00
Tim Rühsen
be7d19f478 Fix iconv conversion
* src/iri.c: Kick out the last converted character from iconv()

Thanks to Eli Zaretskii <eliz@gnu.org> for suggesting the fix.
Reported-by: "Andries E. Brouwer" <Andries.Brouwer@cwi.nl>
2015-12-15 10:55:41 +01:00
Tim Rühsen
358a3c330f Let Test-k survive on CygWin and Windows
* tests/Test-k.px: Use --restrict-file-names for CygWin/Windows
  filename requirements.
2015-12-14 09:36:40 +01:00
Ander Juaristi
478a584609 Fix leak in HSTS code
* src/hsts.c (hsts_store_open): close fp if open.
2015-12-13 16:10:16 +01:00
Ander Juaristi
994c4dcce7 Remove unused variable in ftp code
* src/ftp.c (getftp): fix compiler warning for unused variable.
2015-12-13 16:06:53 +01:00
Giuseppe Scrivano
f65f522f58 Tag release 1.17.1
* NEWS: Update.
* gnulib: sync from upstream.
2015-12-11 18:35:04 +01:00
Tim Rühsen
be84711ea4 Remove ABOUT-NLS from git
* ABOUT-NLS: Remove from repository, it will be autogenerated.
* .gitignore: Add ABOUT-NLS and build-aux/ar-lib
2015-12-11 12:11:55 +01:00
Jernej Simončič
bf56bf4560 * src/metalink.c: Specify 'rb' as mode to open file 2015-12-11 09:58:30 +01:00