Description
cscli hub update consistently logs a warning about a failed HEAD request to cdn-hub.crowdsec.net when the host has dual-stack (IPv4 + IPv6) connectivity. The HEAD request is sent over IPv6 to a CloudFront edge node and gets read: connection reset by peer. The update still succeeds because cscli falls back to GET, but the warning is noisy and confusing.
Reproduction
Server with working dual-stack IPv4/IPv6 connectivity.
$ cscli hub update
WARNING Failed to check last modified: failed to make HEAD request for https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json:
Head "https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json":
read tcp [2a02:121f:1dd:8001::2000]:58536->[2600:9000:202a:4800:b:9c87:4c80:93a1]:443: read: connection reset by peer
Downloading /etc/crowdsec/hub/.index.json
Nothing to do, the hub index is up to date.
Isolated test
# IPv6 HEAD — fails intermittently (~2/3 attempts timeout/RST)
$ for i in 1 2 3; do curl -6 -sI --max-time 5 -X HEAD https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json | head -1; done
(empty)
(empty)
HTTP/2 200
# IPv4 HEAD — always succeeds
$ curl -4 -sI --max-time 5 -X HEAD https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json | head -1
HTTP/2 200
# IPv6 GET — always succeeds
$ curl -6 -sI --max-time 5 https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json | head -1
HTTP/2 200
So the failure is specific to HEAD requests over IPv6 hitting certain CloudFront edges. Different edge nodes are contacted on each retry (different /48 prefixes within 2600:9000:*), and only some respond correctly to HEAD over IPv6.
Environment
- CrowdSec:
v1.7.8-debian-pragmatic-amd64-63227459 (codename alphaga, GoVersion 1.26.2)
- OS: Ubuntu 24.04.4 LTS, kernel 6.8.0-111-generic
- Network: dual-stack, MTU 1500 on
enp6s18, no firewall blocking outbound IPv6
Suspected cause
CloudFront edge configuration inconsistency on IPv6: some edges return HTTP/2 200 for HEAD, others reset the connection. This is upstream (AWS/CloudFront or CrowdSec's CDN config), not a CrowdSec code bug per se.
Suggested mitigations on the cscli side
- Treat HEAD failures as a soft error and fall back to GET silently (current behavior already falls back, but logs a
WARNING that confuses operators).
- Add a
--prefer-ipv4 flag, or document the workaround (/etc/gai.conf precedence override, or /etc/hosts pin to an IPv4 CloudFront address).
- Open an upstream ticket with the CDN provider to fix the inconsistent HEAD-over-IPv6 behavior.
Happy to provide more traces (tcpdump, etc.) if useful.
Description
cscli hub updateconsistently logs a warning about a failed HEAD request tocdn-hub.crowdsec.netwhen the host has dual-stack (IPv4 + IPv6) connectivity. The HEAD request is sent over IPv6 to a CloudFront edge node and getsread: connection reset by peer. The update still succeeds because cscli falls back to GET, but the warning is noisy and confusing.Reproduction
Server with working dual-stack IPv4/IPv6 connectivity.
Isolated test
So the failure is specific to HEAD requests over IPv6 hitting certain CloudFront edges. Different edge nodes are contacted on each retry (different
/48prefixes within2600:9000:*), and only some respond correctly to HEAD over IPv6.Environment
v1.7.8-debian-pragmatic-amd64-63227459(codenamealphaga, GoVersion 1.26.2)enp6s18, no firewall blocking outbound IPv6Suspected cause
CloudFront edge configuration inconsistency on IPv6: some edges return HTTP/2 200 for HEAD, others reset the connection. This is upstream (AWS/CloudFront or CrowdSec's CDN config), not a CrowdSec code bug per se.
Suggested mitigations on the cscli side
WARNINGthat confuses operators).--prefer-ipv4flag, or document the workaround (/etc/gai.confprecedence override, or/etc/hostspin to an IPv4 CloudFront address).Happy to provide more traces (tcpdump, etc.) if useful.