Is your feature request related to a problem? Please describe.
Yes, in two places, both hardcoding an IPv4-only bind:
-
start_metrics_server()
binds via wsgiref.simple_server.make_server("", port, ...), which defaults
socketserver.TCPServer.address_family to socket.AF_INET. On our IPv6-primary cluster,
Istio's sidecar scrapes the pod's IPv6 address and gets connection refused — confirmed via
Envoy access logs (503 delayed_connect_error:_Connection_refused on every scrape), while
curl localhost:8000/metrics from inside the same container succeeds every time.
-
The Feast Operator's
getContainerCommand()
hardcodes -h 0.0.0.0 in the startup command of every service container it renders (online,
ui, offline, lineage), with no CRD field to change it. On the same IPv6-only cluster, none
of these servers bind an address anything can reach.
We work around (2.) today with a Kyverno ClusterPolicy that rewrites each container's command
at admission time — functional, but fragile: it doesn't self-heal already-running Deployments
(skipBackgroundRequests: true), and an earlier version of this exact policy replaced the entire
command with a hardcoded string instead of mapping over the operator's own argv, silently
dropping --workers, --metrics, and other flags for months before we caught it.
Describe the solution you'd like
feast.metrics.start_metrics_server() — bind dual-stack instead of IPv4-only:
import socket
from wsgiref.simple_server import WSGIServer, make_server
class DualStackWSGIServer(WSGIServer):
address_family = socket.AF_INET6
def server_bind(self):
self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
super().server_bind()
httpd = make_server("::", port, app, server_class=DualStackWSGIServer)
Still works on IPv4-only hosts — Linux dual-stack sockets fall back cleanly with
net.ipv6.bindv6only=0, the default.
Feast Operator — add a BindAddress/DualStack field to ServerConfigs (parallel to other
per-service settings) that getContainerCommand() uses instead of the hardcoded 0.0.0.0
literal, accounting for the uvicorn-based ui/lineage servers rejecting a bracketed host (::)
unlike the gunicorn/Flight servers ([::]).
Describe alternatives you've considered
For the Operator half: the Kyverno ClusterPolicy above. It works, but every environment that
needs IPv6 needs Kyverno installed and a policy kept in lockstep with the Operator's own
command-building logic, and a bug in the policy (we've had one) doesn't self-heal already-running
Deployments.
Additional context
Test plan
Is your feature request related to a problem? Please describe.
Yes, in two places, both hardcoding an IPv4-only bind:
start_metrics_server()binds via
wsgiref.simple_server.make_server("", port, ...), which defaultssocketserver.TCPServer.address_familytosocket.AF_INET. On our IPv6-primary cluster,Istio's sidecar scrapes the pod's IPv6 address and gets connection refused — confirmed via
Envoy access logs (
503 delayed_connect_error:_Connection_refusedon every scrape), whilecurl localhost:8000/metricsfrom inside the same container succeeds every time.The Feast Operator's
getContainerCommand()hardcodes
-h 0.0.0.0in the startup command of every service container it renders (online,ui,offline,lineage), with no CRD field to change it. On the same IPv6-only cluster, noneof these servers bind an address anything can reach.
We work around (2.) today with a Kyverno
ClusterPolicythat rewrites each container'scommandat admission time — functional, but fragile: it doesn't self-heal already-running Deployments
(
skipBackgroundRequests: true), and an earlier version of this exact policy replaced the entirecommand with a hardcoded string instead of mapping over the operator's own argv, silently
dropping
--workers,--metrics, and other flags for months before we caught it.Describe the solution you'd like
feast.metrics.start_metrics_server()— bind dual-stack instead of IPv4-only:Still works on IPv4-only hosts — Linux dual-stack sockets fall back cleanly with
net.ipv6.bindv6only=0, the default.Feast Operator — add a
BindAddress/DualStackfield toServerConfigs(parallel to otherper-service settings) that
getContainerCommand()uses instead of the hardcoded0.0.0.0literal, accounting for the uvicorn-based
ui/lineageservers rejecting a bracketed host (::)unlike the gunicorn/Flight servers (
[::]).Describe alternatives you've considered
For the Operator half: the Kyverno
ClusterPolicyabove. It works, but every environment thatneeds IPv6 needs Kyverno installed and a policy kept in lockstep with the Operator's own
command-building logic, and a bug in the policy (we've had one) doesn't self-heal already-running
Deployments.
Additional context
Test plan
start_metrics_server()binds successfully and serves/metricsover both127.0.0.1and::1on a dual-stack host.FeatureStoreCR setting the new bind-address field renders the configured address ineach affected container's command.