[pull] master from curl:master #279
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
CMake builds mbedTLS in Debug mode by default, which was the reason for these consistent test failures: ``` FAIL 1631: 'FTP through HTTPS-proxy' FTP, HTTPS-proxy FAIL 1632: 'FTP through HTTPS-proxy, with connection reuse' FTP, HTTPS-proxy ``` Sometimes also: ``` FAIL 303: 'HTTPS with 8 secs timeout' HTTPS, HTTP GET, timeout, FAILURE ``` https://github.com/curl/curl/actions/runs/11260616621/job/31313234198 Fix it by building in `RelWithDebInfo` mode, matching the bare `Makefile` builds used earlier. (`Release` mode also works.) Cache sizes: - Makefile: 10MB - CMake Release: 1MB - CMake RelWithDebInfo: 2.5MB Ref: #15215 Follow-up to e377c91 #15208 Closes #15238
Previous method extracted this information from the "hack layer" headers' copyright comments. It turns out that the same information can be obtained by querying the default SDK root via a GCC option. According to this test run, they do match: https://github.com/curl/curl/actions/runs/11264283159/job/31323963221?pr=15193 Closes #15241
…ents - use shallow clone for submodules. - reduce total job timeout from 90/60 -> 45 minutes. - use `$HOME` instead of literal. - http3-linux: sync step yaml order with linux.yml. - http3-linux: add cmake + ninja support like in linux.yml. - http3-linux: dump confgure log, test config, curl -V like in linux.yml. - http3-linux: skip restoring gnutls and wolfssl when not used. - dump `curl_config.h`. - fold a long line. Closes #15242
Rust remains installed for the Hyper job, because the rustls cache flag is always false in that case. Closes #15246
…ustls, wolfSSL
Also:
- detect and add required system libraries for Rustls on macOS and
non-Windows.
- add Linux CMake jobs for the touched dependencies.
Caveats:
- MSH3 generates a broken `libmsh3.pc`, so needs manual config.
Upstream PR: nibanks/msh3#225
- Rustls `.pc` file missing, so needs manual config.
An internal change worthy of mention is that we are using the lib path
and name information returned by `pkg-config` as-is. Meaning the libname
doesn't include the full path, like it's usual with native cmake
detection. The path comes separately and needs to be rolled separately.
For this we add it to targets via `link_directories()`. We also keep tab
of them in `CURL_LIBDIRS` and use that in `libcurl.pc`. Feature checks
also need to receive these paths. CMake doesn't offer
a `CMAKE_REQUIRED_*` variable for this purpose, only
a `CMAKE_REQUIRED_LINK_OPTIONS` accepting raw linker flags. Add a macro
to convert a list of paths to linker options to solve it. wolfSSL
requires this for now.
Closes #15193
It's managed by Ninja automatically. Closes #15249
Enclose `CMAKE_EXTRA_INCLUDE_FILES`, `CMAKE_REQUIRED_DEFINITIONS`, `CMAKE_REQUIRED_FLAGS`, `CMAKE_REQUIRED_INCLUDES`, `CMAKE_REQUIRED_LIBRARIES`, `CMAKE_REQUIRED_LINK_OPTIONS`, settings within `cmake_push_check_state()`/`cmake_pop_check_state()` calls. It prevents spilling them into other feature checks. It also replaces manual resets found in some places (which can have the undesired side-effect of destroying values meant for global use.) Cherry-picked from #15157 Closes #15251
…DES` It was done for `zlib`, `brotli`, `libpsl`, `libssh2`, `wolfssh` (a copy-paste case for `wolfssh`). Feature detections should not rely by default on dependency headers. There is no evidence they do now. If it becomes necessary, headers should added for the duration of the feature check. Ref: 118977f Cherry-picked from #15157 Closes #15252
Add comments saying when we want values set in feature check option variables to apply to all feature checks, globally. These are currently: `ws2_32` and `socket` libraries, and `-D_WIN32_WINNT=` macro. Also use `list(APPEND ...)` for the libraries to avoid overwriting potentially existing values. Cherry-picked from #15157 Closes #15253
Replace `check_include_file_concat()` with `check_include_file()` in GSS/LDAP detection to avoid these headers spilling into subsequent feature checks. - For LDAP, reverse detection order to match with `./configure`. Though, in current LDAP packages `ldap.h` does include `lber.h`. - For GSS, align header detection logic with `./configure`, where `gssapi/gssapi_generic.h` might require `gssapi/gssapi.h`, and `gssapi/gssapi_krb5.h` might require both. Ref: #436 Closes #15157
- split the huge Curl_cookie_add() into several smaller static functions - switch to using the common llist instead of custom linked list - use less memory for *getlist() - use bitfields for flags in the Cookie struct - avoid the copy for date parsing - more consistent variable naming Closes #15247
Moved over from transfer.c because it is only used in multi.c Closes #15260
Based on #14135, implement TLSv1.3 earlydata support for the curl command line, libcurl and its implementation in GnuTLS. If a known TLS session announces early data support, and the feature is enabled *and* it is not a "connect-only" transfer, delay the TLS handshake until the first request is being sent. - Add --tls-earldata as new boolean command line option for curl. - Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature. - Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of bytes sent and accepted/rejected by the server. Implementation details: - store the ALPN protocol selected at the SSL session. - When reusing the session and enabling earlydata, use exactly that ALPN protocol for negoptiation with the server. When the sessions ALPN does not match the connections ALPN, earlydata will not be enabled. - Check that the server selected the correct ALPN protocol for an earlydata connect. If the server does not confirm or reports something different, the connect fails. - HTTP/2: delay sending the initial SETTINGS frames during connect, if not connect-only. Verification: - add test_02_32 to verify earlydata GET with nghttpx. - add test_07_70 to verify earlydata PUT with nghttpx. - add support in 'hx-download', 'hx-upload' clients for the feature Assisted-by: ad-chaos on github Closes #15211
A giant function is hard to read. Closes #15250
Homebrew switched to `pkgconf`, and now pkg-config installs an extra package. Update package list to avoid that. The side-effect of `pkgconf` is that this former log message: ``` -- Package 'libcrypto', required by 'libssh2', not found ``` https://github.com/curl/curl/actions/runs/11779568834/job/32808325442#step:7:84 is replaced by this, and repeated 10 times: ``` Package libcrypto was not found in the pkg-config search path. Perhaps you should add the directory containing `libcrypto.pc' to the PKG_CONFIG_PATH environment variable Package 'libcrypto', required by 'libssh2', not found ``` https://github.com/curl/curl/actions/runs/11792711391/job/32846858320#step:7:85 Closes #15575
- Fix the length of strings passed to iconv_open. IBM's iconv_open expects parameters to be a pointer to a 32 byte character array with the unused fields set to 0. Prior to this change, since 8c62479 (precedes 8.11.0), it was incorrectly passed pointers to smaller length const strings and curl would fail with error "blank argument where content is expected". Reported-by: Andrew Kirillov Ref: https://www.ibm.com/docs/en/i/7.5?topic=ssw_ibm_i_75/apis/iconvopn.html Fixes #15570 Closes #15574
An upstream update `impacket` pip package started requiring `blinker`.
An older version is shipping with Ubuntu, causing this on install:
```
Attempting uninstall: blinker
Found existing installation: blinker 1.7.0
ERROR: Cannot uninstall blinker 1.7.0, RECORD file not found. Hint: The package was installed by debian.
```
Fix it by switching to venv and install everything separate from the
system.
The overhead is the same as using `pip --ignore-installed`, which also
installs everything from scratch.
The 3rd option is to uninstall the system `python3-blinker` package, but
it was the slowest.
Closes #15578
`libcurl.pc` `Libs.private` (since 8.11.0, and in `Libs` before 7.20.0) and `curl-config` `--static-libs` (since 7.17.1, and in `Libs` between 7.7.2-7.25.0). This included all flags inherited from the environment, in addition to those coming from dependency detections. To avoid spilling all linker flags inherited from the environment to the libcurl config files, this patch omits them all, except `-L`, `-F`, `--library-path=` and `-framework` options, which are still passed. The rationale for the exceptions is that `LIBS` is passed as-is, and `LDFLAGS`, `LIBS` are the canonical way to pass custom libs options to a build. `LIBS` may not work without a matching custom libpath. This brings autotools behaviour closer to cmake, and `curl-config` closer to `libcurl.pc`. Follow-up to 9f56bb6 #14681 Follow-up to 4c8adc8 Reported-by: Peter Marko Fixes #15533 Closes #15550
krb5 defines `strtok_r` for Windows unconditionally in its public header: https://github.com/krb5/krb5/blob/dc5554394e5a4363b3e109623edbeb9ad6c18a62/src/include/win-mac.h#L214-L215 resulting in this warning: ``` lib\strtok.h(31,9): warning C4005: 'strtok_r': macro redefinition C:\vcpkg\installed\x64-windows\include\win-mac.h(215,9): see previous definition of 'strtok_r' ``` The krb5 macro collides with curl's internal definition, in case the `strtok_r` function is undetected and falling back to a local replacement. Reported-by: Tal Regev Bug: #15549 (comment) Closes #15564
- stop passing explicit libpaths via `CMAKE_SHARED_LINKER_FLAGS` and `CMAKE_EXE_LINKER_FLAGS`. `link_directories()` is doing that already. - use `curl_required_libpaths()` to pass libpaths to the feature test. Reported-by: Daniel Engberg Fixes #15536 Also fixes GSS feature detection with non-gcc/clang compilers, such as MSVC. - add libpaths to `CURL_LIBPATHS`. - move `GSS_CFLAGS`, `GSS_LDFLAGS` stringifications to FindGSS. To match the `CFLAGS` format returned by the rest of Find modules. - reorder calls to match other dependencies. - don't extend system `LDFLAGS` when FindGSS did not return any. - ignore `LDFLAGS` when detecting GSS via `pkg-config`. `LDFLAGS` holds a copy of libpaths and libs in this case. Ignore those to avoid these duplicates making into `libcurl.pc` and `curl-config`. Also syncing behavior with other Find modules which also ignore raw `LDFLAGS`. - ignore raw `LDFLAGS` coming from `krb5-config --libs`. FindGSS no longer returns dependency-specific `LDFLAGS` after this. Syncing behavior with other Find modules. - reduce scope of checker state push/pop/set. Closes #15545
The idea of linking dependencies found to `libcurl.pc` turns out not to work in practice in some cases. Specifically: gss, ldap, mbedtls, libmsh3, rustls A `.pc` may not work or be missing for a couple of reasons: - not all build methods generate it: mbedTLS, Rustls - generated file is broken: msh3 Ref: nibanks/msh3#225 - installed package flavour isn't shipping with one: FreeBSD GSS, OmniOS LDAP, macOS LDAP The effect of such issues shall be subtle in theory, because `libcurl.pc` normally lists these dependencies in the `Requires.private` section meant for static linking. But, e.g. `pkg-config --exists` requires these to be present, and builds sometimes use this check regardless of build type. This bug is not present in `pkgconf`; it only checks for them when `--static` is also passed. Fix these by adding affected `.pc` references to `libcurl.pc` only when we detected the dependency via `pkg-config`. There are a few side-effects of this solution: - references are never added for dependencies where curl doesn't implement `pkg-config` detection. These are: - autotools: ldap, mbedtls, msh3 - cmake: ldap (pending #15273) - generated `libcurl.pc` depends on the build-time environment. - generated `libcurl.pc` depends on curl build tool (cmake, autotools). - generated `libcurl.pc` depends on curl build implementation details. Make an exception for GNU GSS, where I blindly guess that `gss.pc` is always available, as no issues were reported. Other, not mentioned, dependencies continue to be added regardless of the detection method. Reported-by: Harmen Stoppels, Thomas, Daniel Engberg, Andy Fiddaman Fixes #15469 Fixes #15507 Fixes #15535 Fixes #15163 (comment) Closes #15573
This feature was recently dropped because of a bad side-effect of silencing unused cmake command-line option warnings. Fix this issue by retrieving variable values using `get_property()`, instead of accessing the variables directly. It allows restoring this feature without the bad side-effect. Also limit the logic to CI runs. Follow-up to 96edb5f #15501 Closes #15563
With a more portable alternative. Fixes (seen on macOS): ``` ../configure: line 47131: 1: command not found ``` Ref: https://github.com/curl/curl/actions/runs/11846071276/job/33012894013#step:7:635 Follow-up to e244d50 #15550 Closes #15584
- fix socket/sockindex confusion on writes: The callstack used to end with `Curl_write_plain()` accepting a socket till 7.87.0. This call got swapped for `Curl_conn_send()`, expecting a sockindex. `socket_write()` was updated accordingly. Its callers missed it and continued operating on sockets: `do_sec_send()`, `sec_write()`, passing it down the stack and `Curl_conn_send()` resolving it as if it were a sockindex. It affected FTP Kerberos authentication. Discovered through MSVC warnings: ``` curl\lib\krb5.c(652,28): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data curl\lib\krb5.c(654,28): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data curl\lib\krb5.c(656,26): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data curl\lib\krb5.c(657,26): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data curl\lib\krb5.c(665,24): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data curl\lib\krb5.c(666,24): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data ``` Ref: https://github.com/curl/curl/actions/runs/11846599621/job/33014592805#step:9:32 Follow-up to 5651a36 #10280 Bug: #15549 (comment) Fixes #15582 - fix uninitialized buffer: ``` curl\lib\krb5.c(288,1): warning C4701: potentially uninitialized local variable '_gssresp' used ``` Ref: https://github.com/curl/curl/actions/runs/11848626645/job/33020501026?pr=15585#step:9:31 - silence unreachable code compiler warning: ``` curl\lib\krb5.c(370,1): warning C4702: unreachable code ``` Ref: https://github.com/curl/curl/actions/runs/11848626645/job/33020501026?pr=15585#step:9:30 Closes #15585
Not used by libcurl since many years Closes #15538
wolfSSL `--enable-all` builds support ECH. Enable it for 3 jobs using such builds. Also: - GHA/windows: drop ECH from the job name. Closes #15598
- make sure that a match that returns a username also returns a password, that should be blank if no password is found - fix handling of multiple logins for same host where the password/login order might be reversed. - reject credentials provided in the .netrc if they contain ASCII control codes - if the used protocol does not support such (like HTTP and WS do) Reported-by: Harry Sintonen Add test 478, 479 and 480 to verify. Updated unit 1304. Closes #15586
with MSVC 2008 and 2010: ``` lib/netrc.c(107): error C2440: 'initializing' : cannot convert from 'char *' to 'bool' ``` Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51002792/job/jtoxd4mk984oi6fd#L164 Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51002792/job/0wxlw9a8g04e56vt#L177 Follow-up to e9b9bba #15586 Closes #15601
Coverity raised it as a "suspicious sizeof". Closes #15600
Homebrew wolfSSL builds recently enabled all features. It allows to enable ECH in curl for these jobs. Homebrew/homebrew-core@97d1ed6 Homebrew/homebrew-core#191561 Closes #15607
curl offers a `--user-agent` option for modifying the `User-Agent` header supplied in its requests. The man page section for this option explains how to use the `--user-agent` option, but does not explain which `User-Agent` curl uses by default. By default, curl uses curl/VERSION, such as `User-Agent: curl/8.11.0`. Note that this appears to be different from the libcurl default (no User-Agent header). This commit documents the default `User-Agent` in the man page section for the `--user-agent` option, as well as on the "Art of Scripting" page. The `%VERSION` placeholder will be used to insert the current version as described in the man page generator docs. Closes #15608
- Instead of saying "do not use", explain that the function was deprecated for performance reasons. Some users may have a legitimate use of this function even though we deprecated it. Since there are no plans to remove it from the API get rid of the "do not use" warning. Bug: https://curl.se/mail/lib-2024-11/0029.html Reported-by: Jacob Champion Closes #15576
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )