Skip to content

[SERVER] Add Kubernetes engine Pod discovery latency metric - #7722

Open
ruanwenjun wants to merge 3 commits into
apache:masterfrom
ruanwenjun:engine-pod-discovery-latency
Open

ruanwenjun wants to merge 3 commits into
apache:masterfrom
ruanwenjun:engine-pod-discovery-latency

Conversation

@ruanwenjun

@ruanwenjun ruanwenjun commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

Why are the changes needed?

A Spark engine may not be detected by the Kyuubi server until some time after it is created.
This delay can contribute to an engine submission timeout, but the existing engine startup metric only covers engines that register successfully.

Add kyuubi.engine.kubernetes.pod.discovery.latency, a histogram in milliseconds from Pod creation to entry into the informer ADD callback. We use this metric to set an appropriate value for kyuubi.engine.kubernetes.submit.timeout.

How was this patch tested?

Validated with JDK 17. dev/reformat ran without the optional Python formatter (no Python files changed).

Added tests in KubernetesApplicationOperationSuite for ADD latency, repeated ADD events, first UPDATE and DELETE exclusion, invalid timestamps, continued terminal-state handling, and non-engine Pods, without a Kubernetes cluster. All 8 tests passed.

dev/reformat
build/mvn --batch-mode --no-transfer-progress test -pl kyuubi-server -am -Dtest=none -DwildcardSuites=org.apache.kyuubi.engine.KubernetesApplicationOperationSuite -Pflink-provided,hive-provided,spark-provided
git diff --check

Was this patch assisted by generative AI tooling?

Assisted-by: Codex (GPT-6)

@ruanwenjun

Copy link
Copy Markdown
Member Author

@pan3793 PTAL

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant