Skip to content

[sql_server_input_otel, mysql_input_otel]: Add signal_types to system test configs when elastic-package enables dynamic signal testing #17876

Closed
#18658
Closed
[sql_server_input_otel, mysql_input_otel]: Add signal_types to system test configs when elastic-package enables dynamic signal testing#17876
#18658
Assignees
mrodm
Labels
Integration:mysql_input_otelMySQL (OpenTelemetry)Integration:sql_server_input_otelSQL Server OpenTelemetry Input PackageTeam:EcosystemPackages Ecosystem team [elastic/ecosystem]needs:triage

Description

@teresaromero
teresaromero
opened on Mar 18, 2026

Summary

OTel input packages with dynamic_signal_types: true (sql_server_input_otel, mysql_input_otel) have their system tests skipped because elastic-package previously did not support multisignal inputs (elastic/elastic-package#3298). When elastic-package merges PR #3373 (Add dynamic signal types system test support), these packages will need updates so their system tests pass.

The root cause: the pipeline template always defines both logs and metrics pipelines, but log emission is config-driven (disabled by default). When elastic-package discovers data streams and validates documents, it expects both signals. For default test configs, only metrics are emitted, so the test fails when it tries to validate the logs data stream.

Affected packages

Package System test configs Logs enabled by default?
sql_server_input_otel test-default-config.yml, test-datasource-config.yml, test-logs-config.yml No (default), No, Yes
mysql_input_otel test-default-config.yml Yes (query_sample); top_query disabled

Root cause

  1. Pipeline structure (agent/input/input.yml.hbs): Both packages define logs and metrics pipelines unconditionally.
  2. Log emission is optional: enable_query_sample_events and enable_top_query_events default to false in the manifest (sql_server_input_otel); mysql_input_otel has query_sample enabled by default.
  3. elastic-package behavior: With dynamic signal support, elastic-package discovers all data streams from the policy and validates documents in each. For default configs, the logs pipeline exists but no log documents are emitted (sql_server_input_otel).
  4. Result: Test fails because elastic-package expects documents in the logs data stream but finds none.

Proposed solution

Add the signal_types config field (from elastic-package PR #3373) to each system test config to declare which signals the test expects:

Test config Expected signals signal_types
sql_server_input_otel: test-default-config.yml metrics only [metrics]
sql_server_input_otel: test-datasource-config.yml metrics only [metrics]
sql_server_input_otel: test-logs-config.yml logs (and metrics) [logs] or [logs, metrics]
mysql_input_otel: test-default-config.yml logs + metrics [logs, metrics] (or omit to let discovery find both)

Implementation steps

  1. Blocked on: elastic-package PR #3373 merged and released.
  2. Bump: integrations repo elastic-package dependency to a version that includes signal_types support.
  3. sql_server_input_otel:
    • Add signal_types: [metrics] to test-default-config.yml and test-datasource-config.yml.
    • Add signal_types: [logs] (or [logs, metrics]) to test-logs-config.yml.
    • Remove the skip block from all three test configs.
  4. mysql_input_otel:
    • Add signal_types: [logs, metrics] to test-default-config.yml.
    • Remove the skip block from test-default-config.yml.
  5. Verify: Run elastic-package test system for both packages.

References

Activity

  1. teresaromero
    added
    needs:triage
    Integration:mysql_input_otelMySQL (OpenTelemetry)
    Integration:sql_server_input_otelSQL Server OpenTelemetry Input Package
    Team:EcosystemPackages Ecosystem team [elastic/ecosystem]
    on Mar 18, 2026

  2. elasticmachine commented on Mar 18, 2026

    @elasticmachine
    elasticmachine
    on Mar 18, 2026

    Pinging @elastic/ecosystem (Team:Ecosystem)

  3. github-actions commented on Mar 18, 2026

    @github-actions
    github-actionsBot
    on Mar 18, 2026 – with GitHub Actions
    Contributor

    tl;dr: This issue is valid and should be implemented as a single follow-up once elastic-package ships dynamic signal system-test support; right now the repo is still pinned to elastic-package v0.121.0, and only one of four affected system test configs has a draft update in #17794.

    Recommendation

    Proceed with a single consolidated PR after elastic/elastic-package#3373 is merged/released:

    1. bump elastic-package dependency in this repo,
    2. add signal_types to all affected system test configs,
    3. remove current multisignal skip blocks,
    4. run system tests for both packages.

    Findings

    Verified code evidence

    • Dynamic signal templates are enabled in both packages:

      • packages/sql_server_input_otel/manifest.yml:29 → dynamic_signal_types: true
      • packages/mysql_input_otel/manifest.yml:29 → dynamic_signal_types: true

    • Both input templates always define both logs and metrics pipelines:

      • packages/sql_server_input_otel/agent/input/input.yml.hbs:52-58
      • packages/mysql_input_otel/agent/input/input.yml.hbs:59-66

    • SQL Server log event defaults are disabled, making default config effectively metrics-only:

      • packages/sql_server_input_otel/manifest.yml:106-119

    • MySQL defaults enable query-sample logs (true) and disable top-query (false), so default config can emit logs + metrics:

      • packages/mysql_input_otel/manifest.yml:165-177

    • All affected system test configs are still skipped today and do not declare signal_types:

      • packages/sql_server_input_otel/_dev/test/system/test-default-config.yml:1-3
      • packages/sql_server_input_otel/_dev/test/system/test-datasource-config.yml:1-3
      • packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml:1-3
      • packages/mysql_input_otel/_dev/test/system/test-default-config.yml:1-3

    • Current in-flight PR is partial:

      • #17794 modifies only packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml (removes skip, adds signal_types: [logs]).

    • Tooling readiness:

      • go.mod:9 pins github.com/elastic/elastic-package v0.121.0.
      • elastic/elastic-package#3373 is still open (not merged/released yet).

    Verification

    Commands run and output 
    $ grep -n "dynamic_signal_types: true" packages/sql_server_input_otel/manifest.yml packages/mysql_input_otel/manifest.yml
packages/sql_server_input_otel/manifest.yml:29:    dynamic_signal_types: true
packages/mysql_input_otel/manifest.yml:29:    dynamic_signal_types: true

$ grep -nE "^skip:|signal_types|enable_query_sample_events|enable_top_query_events" packages/sql_server_input_otel/_dev/test/system/*.yml packages/mysql_input_otel/_dev/test/system/*.yml
packages/sql_server_input_otel/_dev/test/system/test-datasource-config.yml:1:skip:
packages/sql_server_input_otel/_dev/test/system/test-default-config.yml:1:skip:
packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml:1:skip:
packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml:12:  enable_query_sample_events: true
packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml:13:  enable_top_query_events: true
packages/mysql_input_otel/_dev/test/system/test-default-config.yml:1:skip:

$ grep -n "github.com/elastic/elastic-package" go.mod
9:	github.com/elastic/elastic-package v0.121.0

    I did not run elastic-package test system yet because the required upstream signal_types support (elastic-package#3373) is still open.

    Detailed Action Plan

    Implementation plan with file targets

    1. Wait for upstream support release

      • Confirm elastic/elastic-package#3373 is merged and released.

    2. Bump dependency

      • Update go.mod (current pin at go.mod:9) to the first version including signal_types system-test support.

    3. Update SQL Server system tests

      • packages/sql_server_input_otel/_dev/test/system/test-default-config.yml
        • Remove skip block at :1-3
        • Add signal_types: [metrics]
      • packages/sql_server_input_otel/_dev/test/system/test-datasource-config.yml
        • Remove skip block at :1-3
        • Add signal_types: [metrics]
      • packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml
        • Remove skip block at :1-3
        • Keep/add signal_types for log validation ([logs] or [logs, metrics] depending on desired assertion scope)

    4. Update MySQL system test

      • packages/mysql_input_otel/_dev/test/system/test-default-config.yml
        • Remove skip block at :1-3
        • Add signal_types: [logs, metrics]

    5. Validate

      • Run system tests for both packages (and align expected fields if any assertions fail):
        • elastic-package test system in packages/sql_server_input_otel
        • elastic-package test system in packages/mysql_input_otel

    6. Close coordination gap with Update sql_server_input_otel logs signal test #17794

      • Either expand #17794 to include all remaining files above, or open a follow-up PR explicitly covering the three still-skipped configs.

    Related Items

    Issues, PRs, and files
    Type Link Relevance
    Issue #17876 Tracking issue for this work
    PR #17794 Partial draft implementation (sql_server logs test only)
    External PR elastic/elastic-package#3373 Adds signal_types system-test support
    External Issue elastic/elastic-package#3298 Parent multisignal system-testing need
    File go.mod:9 Current elastic-package pin (v0.121.0)
    File packages/sql_server_input_otel/_dev/test/system/test-default-config.yml:1 Currently skipped
    File packages/sql_server_input_otel/_dev/test/system/test-datasource-config.yml:1 Currently skipped
    File packages/sql_server_input_otel/_dev/test/system/test-logs-config.yml:1 Currently skipped
    File packages/mysql_input_otel/_dev/test/system/test-default-config.yml:1 Currently skipped

    What is this? | From workflow: Issue Triage

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  4. mrodm
    self-assigned this
    on Apr 24, 2026

  5. mrodm commented on Apr 27, 2026

    @mrodm
    mrodm
    on Apr 27, 2026
    Collaborator

    As part of this issue it would be good to update the test packages in elastic-package for sql_server_input_otel and mysql_input_otel to match the system and policy tests included here. This would help us to validate in elastic-package these kind of packages too.

  6. mrodm commented on Apr 28, 2026

    @mrodm
    mrodm
    on Apr 28, 2026
    Collaborator

    PRs created for this issue:

  7. mrodm
    closed this as completedin #18658on Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Integration:mysql_input_otelMySQL (OpenTelemetry)Integration:sql_server_input_otelSQL Server OpenTelemetry Input PackageTeam:EcosystemPackages Ecosystem team [elastic/ecosystem]needs:triage

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions