Closes #1452 - Implemented Prometheus metrics and ServiceMonitor support for mongodb - #1521

Good work and thank you for the support! I will do the review and local tests.
What I also had in mind is using another existing chart (https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-mongodb-exporter) as dependency, but not sure if this is a good idea.

What I'm also not sure about is the monitoring of arbiter or hidden secondary instances.

Thanks @groundhog2k! Both questions are good ones — and the second one actually surfaced a bug, which I've just pushed a fix for (2b29cb6f).

1. Sidecar vs. prometheus-mongodb-exporter as a dependency

My vote is to keep the sidecar, for reasons that are fairly specific to this chart:

• Per-member metrics need a per-member exporter. mongodb_exporter reports the state of the node it is connected to (replSetGetStatus, oplog window, member state, lag). The dependency chart deploys a single Deployment against one URI, so in a replica set you'd either get metrics from whichever member the driver picked, or you'd have to instantiate the subchart N times. The sidecar scales 1:1 with replicaSet.secondaries for free.
• Credentials stay in the pod. The sidecar talks over localhost and reuses this chart's existing secret via envFrom. A separate Deployment needs its own copy of the root credentials, or a hardcoded cross-chart secret reference.
• No new dependency. The charts in this repo are dependency-free today, and a subchart drags in its own values namespace, upgrade cadence, and image-pinning conventions that wouldn't match the rest of charts/mongodb.

The two aren't mutually exclusive, though: metrics.exporter.uri lets anyone point at an external MongoDB, and since the whole block is metrics.enabled: false by default, users who prefer the standalone chart can just run it alongside. So the sidecar is the turnkey path, not a lock-in.

2. Arbiter and hidden secondaries — you were right to ask

The chart has three StatefulSets (statefulset.yaml, statefulset-arbiter.yaml, statefulset-hidden.yaml), and every Service in the chart discriminates between them with service-type: primary-secondary|arbiter|hidden-secondary. My service-metrics.yaml selector was missing that label, so it matched arbiter and hidden pods too — pods that have no exporter container and no exporter port. Fixed in 2b29cb6f; the metrics Service now scopes to service-type: primary-secondary like service-internal.yaml does. helm lint is green with replicaSet.enabled=true, arbiter.enabled=true, hiddenSecondaries.instances=1.

On whether to extend coverage — my suggestion:

• Hidden secondaries: worth doing. They're fully data-bearing, so the entire metric surface is meaningful, and lag on a hidden member (usually a backup/analytics node) is exactly what you want alerts on. It's the same sidecar block in statefulset-hidden.yaml plus a matching Service/ServiceMonitor.
• Arbiter: I'd skip it. An arbiter holds no data, so the data-oriented collectors (dbstats, collstats, indexstats, topmetrics) return nothing or error out. The only useful signal is its member state — and that's already reported by the primary's replSetGetStatus, which enumerates every member including arbiters. A dedicated arbiter exporter would mostly add a failing-scrape target.

One related thing worth deciding: the default URI is mongodb://…@localhost:<port>/admin?authSource=admin with no directConnection=true. Without it the driver can do replica-set discovery and route reads to the primary, which would make every member's sidecar report the primary's numbers instead of its own. Happy to add directConnection=true to the default URI (still overridable via metrics.exporter.uri) if you agree that's the right default.

Let me know which way you want to go on hidden-secondary coverage and I'll push it in the same PR.

@somaz94
Yes, please add metrics support for the hidden secondaries too and I also agree to the directConnection=true idea. Let's collect the metrics of all the pods that keep data.
If anyone wants only the "primary monitoring", than a changed metrics.exporter.uri can be used or the external mongodb_exporter chart approach.

Thank you very much for your help!

Pushed in 2fe176dd. Both are in:

• Hidden secondaries now run the same exporter sidecar (statefulset-hidden.yaml), with their own <release>-mongodb-metrics-hidden Service + ServiceMonitor scoped to service-type: hidden-secondary. Arbiters are left out as discussed — they hold no data, and their member state already shows up in the primary's replSetGetStatus.
• directConnection=true is now pinned on the auto-built URI (both the authenticated and the no-auth form), so each sidecar reports its own node instead of being routed to the primary by replica-set discovery. Still fully overridable via metrics.exporter.uri.

One small design note: the hidden metrics Service inherits metrics.service.type but omits any fixed nodePort / loadBalancerIP, since those can only belong to one Service once both the primary and hidden Services exist.

Validated locally — ct lint (schema + yamllint + helm lint) green, and helm template across the metrics-off / no-auth / root-credential scenarios renders the exporter on both StatefulSets, both metrics Services, both ServiceMonitors, and the directConnection=true URIs. Thanks!

Thank you! I will take time for a review and test.

I deployed a local kube-prometheus-stack setup and also a Mongodb with metrics support and at the moment I see alerts because of missing metrics. I will continue verifing this tomorrow.

Thanks for testing it locally — the missing-metrics alerts you saw are a real bug in this PR, and I think I've found the cause. It wasn't the Service/ServiceMonitor wiring; it was that the exporter wasn't collecting anything.

metrics.exporter.args defaulted to [], and in percona/mongodb_exporter every --collector.* flag is opt-in. Started with no arguments the sidecar exposes only mongodb_up — so the scrape target looks perfectly healthy while essentially no MongoDB metrics exist, which is exactly the shape of the alerts you described.

Measured against a live replica set with exporter 0.51.0:

mongodb_ss_connections, mongodb_ss_opcounters, mongodb_ss_mem_resident and mongodb_rs_members_state were all absent on the old default.

311e8e40 changes the default to --collector.diagnosticdata (serverStatus) + --collector.replicasetstatus (replSetGetStatus), which is what standard MongoDB dashboards consume, and documents it in the README. I deliberately did not default to --collect-all: it also enables $collStats and $indexStats, whose series count grows with the number of user collections — it already produced 6,092 series against an empty database. Users who want that detail can append the extra collector flags or switch to --collect-all.

One thing worth knowing when you re-test: the exporter needs a warm-up scrape. The very first request to /metrics returns almost nothing even when correctly configured, so a single curl right after startup is misleading — scrape it twice.

Could you re-run your kube-prometheus-stack setup against this commit and confirm the alerts clear? If anything is still missing after this, it's likely a separate issue and I'd like to see which metric names your alerts reference.

Thank you!!