Skip to content

ssi: add processor tags and preserve_original_event on failure (2/4) - #20573

Merged
kcreddy merged 11 commits into
elastic:mainfrom
kcreddy:tagging-evt-original-standards-2
Aug 21, 2026
Merged

kcreddy merged 11 commits into
elastic:mainfrom
kcreddy:tagging-evt-original-standards-2

Conversation

@kcreddy

@kcreddy kcreddy commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Proposed commit message

ssi: add processor tags and preserve_original_event on failure (2/4)

Tag every ingest pipeline processor across 51 SSI-owned packages
(eset_protect through sentinel_one_cloud_funnel), so that failure
telemetry can attribute an error to the step that produced it rather
than collapsing same-type processors into one bucket. All 24011
processors across the 341 pipeline files now carry a tag, 13136 of them
newly, including those nested inside `on_failure` handlers and
`foreach` bodies, and those in the pipeline-level `on_failure` block.

Every processor gets a tag of the form `<descriptive_name>_<8hex>`,
where the 8-hex suffix is a content hash over the processor body and
its enclosing context. Identical constructs in identical surroundings
produce the same hash in every package, giving each tag a stable global
identity in failure telemetry. Tags that already carried a hash are
left byte-identical -- 1293 of them, in sentinel_one, m365_defender and
okta, which shipped tags before this series; a hash is only meaningful
relative to the generator that produced it, so re-hashing a published
tag churns a value consumers may key off. Those three packages therefore
keep the earlier suffix format. Tags that would have said nothing
(`script_<hash>`, `fail_<hash>`) are seeded from the processor's
description, name, message or preceding comment, and tags whose leading
word named the wrong action (`set_...` on an append) are corrected.

Add preserve_original_event to pipeline-level on_failure handlers that
were missing it, ensuring the raw payload is retained when a pipeline
error document is indexed. Thirteen of the 51 packages needed it.

Pipeline-level on_failure error.message appenders use the conventional
four-line template so that the processor tag is captured in the error
record: processor type, a conditional tag line, and the failure
message. The template uses a folded block scalar (`value: >-`) rather
than a literal block (`value: |-`). Long `if:` conditions in
google_workspace/admin and sentinel_one_cloud_funnel pipelines are
reformatted using `if: >-` for readability.

Seventeen processors across nine packages were exact duplicates of an
earlier sibling and could never have any effect; they are removed
rather than given a disambiguating tag. Two further fixes ride along,
each in the affected package's changelog:

  - gcp, jamf_protect: six pipeline-level on_failure handlers set
    error.message instead of appending, discarding what processor-level
    handlers recorded. Only jamf_protect's telemetry_legacy audit
    pipeline records any today, but the pattern is wrong in all six.
    error.message is now an array on failed documents.

These are enhancements, so the packages take a minor version bump.
sailpoint_identity_sc keeps its 2.0.0-next prerelease and takes its
changelog entry there instead.

Updates #20558

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Related issues

Tag every ingest pipeline processor across 52 SSI-owned packages
(entityanalytics_okta through sentinel_one_cloud_funnel) with a
unique, descriptive identifier so that failure telemetry can
attribute errors to the specific step that failed rather than
collapsing same-type processors into one bucket.

Add preserve_original_event to pipeline-level on_failure handlers
that were missing it, ensuring the raw payload is retained when a
pipeline error document is indexed.

Updates elastic#20558
@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Elastic Docs Style Checker (Vale)

Summary: 139 warnings, 94 suggestions found

⚠️ Warnings (139): Fix when the suggestion improves clarity or correctness.
File Line Rule Message
packages/eset_protect/changelog.yml 132 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/eset_protect/changelog.yml 142 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/eset_protect/manifest.yml 90 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/first_epss/changelog.yml 64 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/first_epss/manifest.yml 49 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/forgerock/changelog.yml 54 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/gcp/changelog.yml 331 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/github/changelog.yml 204 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/github/changelog.yml 410 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/github/manifest.yml 96 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/gitlab/changelog.yml 67 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/gitlab/changelog.yml 77 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_scc/changelog.yml 100 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_scc/changelog.yml 110 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_scc/manifest.yml 121 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/google_secops/manifest.yml 82 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/google_workspace/changelog.yml 224 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_workspace/changelog.yml 257 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_workspace/changelog.yml 262 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/google_workspace/changelog.yml 440 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/google_workspace/manifest.yml 105 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/google_workspace/manifest.yml 122 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/google_workspace/manifest.yml 177 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/google_workspace/manifest.yml 194 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/ibm_qradar/manifest.yml 61 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/imperva_cloud_waf/changelog.yml 137 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/imperva_cloud_waf/changelog.yml 147 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/infoblox_bloxone_ddi/changelog.yml 39 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/infoblox_bloxone_ddi/changelog.yml 164 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/infoblox_bloxone_ddi/manifest.yml 71 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/island_browser/manifest.yml 76 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/jamf_compliance_reporter/changelog.yml 29 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/jamf_pro/changelog.yml 140 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/jamf_protect/changelog.yml 102 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/jamf_protect/manifest.yml 191 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/jumpcloud/changelog.yml 80 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/keycloak/changelog.yml 59 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/keycloak/changelog.yml 174 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/lastpass/changelog.yml 49 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/lastpass/changelog.yml 170 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/lastpass/manifest.yml 75 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/lumos/changelog.yml 44 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/lyve_cloud/changelog.yml 24 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/lyve_cloud/changelog.yml 109 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/m365_defender/changelog.yml 61 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 61 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 71 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 71 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 71 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 71 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 157 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 335 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 350 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/m365_defender/changelog.yml 598 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/m365_defender/manifest.yml 97 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/m365_defender/manifest.yml 197 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/mattermost/changelog.yml 27 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/mattermost/changelog.yml 127 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/menlo/changelog.yml 55 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/menlo/changelog.yml 65 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/menlo/manifest.yml 64 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/microsoft_defender_cloud/changelog.yml 67 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_cloud/changelog.yml 77 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_cloud/manifest.yml 108 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 93 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 238 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_defender_endpoint/changelog.yml 358 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/microsoft_defender_endpoint/manifest.yml 104 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/microsoft_exchange_online_message_trace/changelog.yml 132 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_exchange_online_message_trace/changelog.yml 300 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/microsoft_exchange_online_message_trace/manifest.yml 121 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/microsoft_sentinel/changelog.yml 105 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_sentinel/changelog.yml 110 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/microsoft_sentinel/manifest.yml 116 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/mimecast/changelog.yml 186 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/mimecast/changelog.yml 349 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/mimecast/manifest.yml 7 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/miniflux/manifest.yml 64 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/netskope/changelog.yml 95 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/netskope/changelog.yml 200 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/netskope/changelog.yml 263 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/netskope/changelog.yml 263 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/netskope/manifest.yml 138 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/okta/changelog.yml 119 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/okta/changelog.yml 302 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/okta/manifest.yml 220 Elastic.Latinisms Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/opencanary/changelog.yml 17 Elastic.BritishSpellings Use American English spelling 'utilize' instead of British English 'Utilise'.
packages/opencanary/changelog.yml 71 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/panw_cortex_xdr/changelog.yml 94 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/panw_cortex_xdr/changelog.yml 234 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/ping_one/changelog.yml 54 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/ping_one/changelog.yml 159 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/ping_one/manifest.yml 129 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/pps/changelog.yml 66 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/prisma_cloud/changelog.yml 108 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/prisma_cloud/changelog.yml 118 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/prisma_cloud/manifest.yml 90 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/proofpoint_essentials/manifest.yml 75 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/proofpoint_on_demand/changelog.yml 82 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/proofpoint_on_demand/changelog.yml 92 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/proofpoint_tap/changelog.yml 59 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/proofpoint_tap/changelog.yml 202 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/proofpoint_tap/manifest.yml 70 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/qualys_vmdr/changelog.yml 200 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/qualys_vmdr/changelog.yml 215 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/qualys_vmdr/changelog.yml 334 Elastic.BritishSpellings Use American English spelling 'behavior' instead of British English 'behaviour'.
packages/qualys_vmdr/manifest.yml 74 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/rapid7_insightvm/changelog.yml 103 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/rapid7_insightvm/changelog.yml 113 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/rapid7_insightvm/manifest.yml 83 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/rapid7_insightvm/manifest.yml 139 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/santa/changelog.yml 24 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/santa/changelog.yml 119 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/sentinel_one/changelog.yml 266 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/sentinel_one/changelog.yml 411 Elastic.DontUse Don't use 'and/or'. Choose a more precise or reader-focused term.
packages/sentinel_one/manifest.yml 92 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/sentinel_one/manifest.yml 154 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
packages/sentinel_one_cloud_funnel/changelog.yml 41 Elastic.BritishSpellings Use American English spelling 'behavior' instead of British English 'behaviour'.
packages/sentinel_one_cloud_funnel/changelog.yml 86 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/sentinel_one_cloud_funnel/manifest.yml 82 Elastic.Latinisms Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/sentinel_one_cloud_funnel/manifest.yml 149 Elastic.DontUse Don't use 'Please'. Choose a more precise or reader-focused term.
💡 Suggestions (94): Optional style improvements. Apply when helpful.
File Line Rule Message
packages/eset_protect/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/eset_protect/changelog.yml 41 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/first_epss/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/first_epss/changelog.yml 34 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/forgerock/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/gcp/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/github/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/github/changelog.yml 85 Elastic.Wordiness Consider using 'use' instead of 'utilize'.
packages/github/changelog.yml 139 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/gitlab/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/google_scc/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/google_scc/changelog.yml 56 Elastic.Wordiness Consider using 'per' instead of 'as per'.
packages/google_secops/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/google_secops/changelog.yml 36 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/google_secops/manifest.yml 52 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/google_workspace/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/google_workspace/changelog.yml 84 Elastic.Wordiness Consider using 'per' instead of 'as per'.
packages/google_workspace/changelog.yml 129 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/google_workspace/manifest.yml 121 Elastic.Wordiness Consider using 'impossible' instead of 'not possible'.
packages/google_workspace/manifest.yml 193 Elastic.Wordiness Consider using 'impossible' instead of 'not possible'.
packages/greenhouse/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/ibm_qradar/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/imperva_cloud_waf/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/imperva_cloud_waf/changelog.yml 34 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/infoblox_bloxone_ddi/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/island_browser/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/jamf_compliance_reporter/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/jamf_pro/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/jamf_protect/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/jamf_protect/manifest.yml 176 Elastic.Wordiness Consider using 'before' instead of 'prior to'.
packages/jumpcloud/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/keycloak/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/lastpass/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/lumos/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/lyve_cloud/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/m365_defender/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/m365_defender/changelog.yml 19 Elastic.Wordiness Consider using 'remove' instead of 'eliminate'.
packages/m365_defender/changelog.yml 36 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
packages/m365_defender/changelog.yml 210 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/m365_defender/manifest.yml 32 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/mattermost/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/menlo/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_defender_cloud/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_defender_endpoint/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_defender_endpoint/changelog.yml 29 Elastic.Wordiness Consider using 'remove' instead of 'eliminate'.
packages/microsoft_defender_endpoint/changelog.yml 143 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/microsoft_defender_endpoint/manifest.yml 32 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/microsoft_exchange_online_message_trace/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_intune/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_sentinel/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/microsoft_sentinel/changelog.yml 50 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/mimecast/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/mimecast/changelog.yml 15 Elastic.WordChoice Consider using 'deactivated, deselected, hidden, turned off, unavailable' instead of 'disabled', unless the term is in the UI.
packages/mimecast/changelog.yml 98 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/miniflux/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/miniflux/changelog.yml 37 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/netskope/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/okta/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/okta/manifest.yml 104 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/okta/manifest.yml 115 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/okta/manifest.yml 125 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/okta/manifest.yml 203 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'Disable', unless the term is in the UI.
packages/okta/manifest.yml 204 Elastic.WordChoice Consider using 'deactivated, deselected, hidden, turned off, unavailable' instead of 'disabled', unless the term is in the UI.
packages/opencanary/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/panw_cortex_xdr/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/panw_cortex_xdr/changelog.yml 39 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/ping_federate/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/ping_one/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/pps/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/prisma_cloud/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/proofpoint_essentials/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/proofpoint_itm/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/proofpoint_itm/changelog.yml 29 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/proofpoint_on_demand/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/proofpoint_tap/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/proofpoint_tap/manifest.yml 3 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 5 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 15 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 20 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 24 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 38 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 39 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 44 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'TAP'.
packages/proofpoint_tap/manifest.yml 44 Elastic.DeviceAgnosticism Use device-agnostic language when possible. Use 'select' instead of 'tap'.
packages/qualys_vmdr/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/qualys_vmdr/changelog.yml 112 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/qualys_vmdr/changelog.yml 339 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'Disable', unless the term is in the UI.
packages/rapid7_insightvm/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/rapid7_insightvm/changelog.yml 77 Elastic.WordChoice Consider using 'stop, exit' instead of 'terminate', unless the term is in the UI.
packages/rapid7_insightvm/manifest.yml 46 Elastic.WordChoice Consider using 'cannot' instead of 'unable', unless the term is in the UI.
packages/sailpoint_identity_sc/changelog.yml 12 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'Disable', unless the term is in the UI.
packages/santa/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/sentinel_one/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/sentinel_one_cloud_funnel/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

kcreddy added 2 commits August 6, 2026 19:44
Update the link field in each new changelog entry from the tracking
issue to the merged pull request.

Updates elastic#20558
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

Tag every ingest pipeline processor across 52 SSI-owned packages
(entityanalytics_okta through sentinel_one_cloud_funnel) with a unique,
descriptive identifier so that failure telemetry can attribute an error
to the step that produced it rather than collapsing same-type
processors into one bucket. `_ingest.on_failure_processor_tag` is only
useful when the tag is both present and unique, so all 23880 processors
across the 343 pipeline files now carry one -- including those nested
inside `on_failure` handlers and `foreach` bodies, and those in the
pipeline-level `on_failure` block, which are the easiest to overlook.

Tags that already existed are kept as they are. They are quoted in
error.message strings, dashboards and runbooks, so renaming one is a
breaking change for whatever reads it. Where a name was already used by
more than one processor in the same pipeline, the first occurrence
keeps it and the later ones take a short hash suffix; a handful of tags
written as free text ("Process DNS RData") become identifiers.

Add preserve_original_event to pipeline-level on_failure handlers that
were missing it, ensuring the raw payload is retained when a pipeline
error document is indexed. Twelve of the 52 packages needed it.

Seventeen processors were exact duplicates of an earlier sibling and so
could never have any effect. Remove them rather than give them a
disambiguating tag; each affected package records the removal in its
changelog.

Updates elastic#20558
Tag every ingest pipeline processor across 51 SSI-owned packages
(eset_protect through sentinel_one_cloud_funnel), so that failure
telemetry can attribute an error to the step that produced it rather
than collapsing same-type processors into one bucket. All 24011
processors across the 341 pipeline files now carry a tag, 13136 of them
newly, including those nested inside `on_failure` handlers and
`foreach` bodies, and those in the pipeline-level `on_failure` block.

entityanalytics_okta is no longer part of this batch: upstream elastic#20212
tagged its pipelines and added the missing on_failure handlers first,
so there was nothing left to change.

Existing tags keep their name. Where a name was not already unique in
its pipeline, or the processor had no tag at all, it gains an 8-hex
suffix hashed over the processor's content and everything enclosing it.
Tags that already carried a hash are left byte-identical -- 1293 of
them, in sentinel_one, m365_defender and okta, which shipped tags
before this series; a hash is only meaningful relative to the generator
that produced it, so re-hashing a published tag churns a value
consumers may key off. Those three packages therefore keep the earlier
suffix format. Tags that would have said nothing (`script_<hash>`,
`fail_<hash>`) are seeded from the processor's description, name,
message or preceding comment, and tags whose leading word named the
wrong action (`set_...` on an append) are corrected.

Add preserve_original_event to pipeline-level on_failure handlers that
were missing it, ensuring the raw payload is retained when a pipeline
error document is indexed. Twelve of the 51 packages needed it.

Seventeen processors across nine packages were exact duplicates of an
earlier sibling and could never have any effect; they are removed
rather than given a disambiguating tag. One further fix rides along,
recorded in the affected packages' changelogs:

  - gcp, jamf_protect: six pipeline-level on_failure handlers set
    error.message instead of appending, discarding what processor-level
    handlers recorded. Only jamf_protect's telemetry_legacy audit
    pipeline records any today, but the pattern is wrong in all six.
    error.message is now an array on failed documents.

These are enhancements, so the packages take a minor version bump.
sailpoint_identity_sc keeps its 2.0.0-next prerelease and takes its
changelog entry there instead.

Updates elastic#20558
@kcreddy kcreddy self-assigned this Aug 18, 2026
@kcreddy
kcreddy marked this pull request as ready for review August 18, 2026 17:53
Copilot AI lite review requested due to automatic review settings August 18, 2026 17:53
@kcreddy
kcreddy requested review from a team as code owners August 18, 2026 17:53
@kcreddy kcreddy added enhancement New feature or request Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Aug 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy

kcreddy commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

The failing CI failure is due to google_scc issue #20792 (comment) should be fixed by elastic/stream#213 and #20815

Comment thread packages/first_epss/changelog.yml Outdated
Comment thread packages/gcp/data_stream/cloudsql_mysql/elasticsearch/ingest_pipeline/default.yml Outdated
@efd6

efd6 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

There are many cases of

  - append:
      tag: append_error_message_206bb8ac
      field: error.message
      value: '{{{ _ingest.on_failure_message }}}'

These should be extended to the conventional form so that the processor tag values get into the error message.

Copilot AI review requested due to automatic review settings August 20, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

💔 Build Failed

Failed CI Steps

History

cc @kcreddy

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package lumos - 1.9.0 containing this change is available at https://epr.elastic.co/package/lumos/1.9.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package lyve_cloud - 1.18.0 containing this change is available at https://epr.elastic.co/package/lyve_cloud/1.18.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package m365_defender - 5.16.0 containing this change is available at https://epr.elastic.co/package/m365_defender/5.16.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package mattermost - 2.6.0 containing this change is available at https://epr.elastic.co/package/mattermost/2.6.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package menlo - 1.9.0 containing this change is available at https://epr.elastic.co/package/menlo/1.9.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_defender_cloud - 3.6.0 containing this change is available at https://epr.elastic.co/package/microsoft_defender_cloud/3.6.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_defender_endpoint - 4.10.0 containing this change is available at https://epr.elastic.co/package/microsoft_defender_endpoint/4.10.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_exchange_online_message_trace - 2.3.0 containing this change is available at https://epr.elastic.co/package/microsoft_exchange_online_message_trace/2.3.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_intune - 0.2.0 containing this change is available at https://epr.elastic.co/package/microsoft_intune/0.2.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_sentinel - 1.5.0 containing this change is available at https://epr.elastic.co/package/microsoft_sentinel/1.5.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package mimecast - 3.7.0 containing this change is available at https://epr.elastic.co/package/mimecast/3.7.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package miniflux - 1.3.0 containing this change is available at https://epr.elastic.co/package/miniflux/1.3.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package netskope - 3.2.0 containing this change is available at https://epr.elastic.co/package/netskope/3.2.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package okta - 3.16.0 containing this change is available at https://epr.elastic.co/package/okta/3.16.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package opencanary - 1.1.0 containing this change is available at https://epr.elastic.co/package/opencanary/1.1.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package panw_cortex_xdr - 2.8.0 containing this change is available at https://epr.elastic.co/package/panw_cortex_xdr/2.8.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package ping_federate - 1.3.0 containing this change is available at https://epr.elastic.co/package/ping_federate/1.3.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package ping_one - 1.25.0 containing this change is available at https://epr.elastic.co/package/ping_one/1.25.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package pps - 1.3.0 containing this change is available at https://epr.elastic.co/package/pps/1.3.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package prisma_cloud - 4.3.0 containing this change is available at https://epr.elastic.co/package/prisma_cloud/4.3.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package proofpoint_essentials - 1.2.0 containing this change is available at https://epr.elastic.co/package/proofpoint_essentials/1.2.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package proofpoint_itm - 1.2.0 containing this change is available at https://epr.elastic.co/package/proofpoint_itm/1.2.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package proofpoint_on_demand - 1.11.0 containing this change is available at https://epr.elastic.co/package/proofpoint_on_demand/1.11.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package proofpoint_tap - 1.32.0 containing this change is available at https://epr.elastic.co/package/proofpoint_tap/1.32.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package qualys_vmdr - 6.21.0 containing this change is available at https://epr.elastic.co/package/qualys_vmdr/6.21.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package rapid7_insightvm - 2.10.0 containing this change is available at https://epr.elastic.co/package/rapid7_insightvm/2.10.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package santa - 3.25.0 containing this change is available at https://epr.elastic.co/package/santa/3.25.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package sentinel_one - 2.12.0 containing this change is available at https://epr.elastic.co/package/sentinel_one/2.12.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package sentinel_one_cloud_funnel - 1.15.0 containing this change is available at https://epr.elastic.co/package/sentinel_one_cloud_funnel/1.15.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package sailpoint_identity_sc - 2.0.0 containing this change is available at https://epr.elastic.co/package/sailpoint_identity_sc/2.0.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants