Skip to content

[CrowdStrike] Add support for NG-SIEM in alert pipeline #19048

Closed
Enhancement
#19494
Closed
[CrowdStrike] Add support for NG-SIEM in alert pipeline#19048
Enhancement
#19494
Assignees
navnit-elastic
Labels
Integration:crowdstrikeCrowdStrikeTeam:SDE-CrestCrest developers on the Security Integrations team [elastic/sit-crest-contractors]Team:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]enhancementNew feature or request

Description

@jamiehynds
jamiehynds
opened on May 18, 2026

Summary
The alert data stream pipeline currently handles EPP (endpoint protection) style alerts from the CrowdStrike Alerts REST API. CrowdStrike's NG-SIEM product generates a distinct alert type — correlation-detection — that shares only a handful of field names with EPP alerts and exposes host, user, IP, and threat context via different top-level fields. These events pass through the current pipeline largely unprocessed, resulting in missing event.severity, host.name, user.name, and other fields.

NG-SIEM correlation-detection events are identifiable by product: "ngsiem" and type: "correlation-detection". Sample events are available on request.

Problem
When a NG-SIEM correlation-detection event is ingested through the current pipeline, only 11 of its ~50 fields are processed (id, agent_id, aggregate_id, cid, composite_id, crawled_timestamp, created_timestamp, data_domains, description, falcon_host_link, name). All of the following are silently dropped into unprocessed json.* fields:

  • severity / severity_name → event.severity is never set
  • status → crowdstrike.alert.status is never set
  • type: "correlation-detection" → not captured anywhere
  • host_names[] → no host.name (pipeline expects EPP-style device.hostname)
  • source_ips[] → no host.ip or related.ip
  • user_names[] / users[] → no user.name or related.user
  • mitre_attack[] → no threat.* mappings
  • All correlation_rule_* fields → lost entirely

Additionally, unlike other alert pipelines (e.g. Microsoft Sentinel), the CrowdStrike alert pipeline does not remove the temporary json.* object at the end of processing, meaning unprocessed fields accumulate in the document.

Suggested approach
The following mappings are suggestions based on analysis of sample events . They should be validated against sample data and discussed with the team before implementation. Sample events are available on request.

Timestamps

  • updated_timestamp → crowdstrike.alert.updated_timestamp
  • start_time → event.start + crowdstrike.alert.start_time
  • end_time → event.end + crowdstrike.alert.end_time
  • Compute event.duration in nanoseconds from event.start and event.end

Event categorisation

  • severity → event.severity directly (already an integer in the 0–100 range per the CrowdStrike Parsing Standard — no script transformation needed, unlike the Sentinel pipeline)
  • severity_name, status, type, product → crowdstrike.alert.*

Host context

  • host_names[0] → host.name; full array → crowdstrike.alert.host_names + related.hosts
  • source_ips → crowdstrike.alert.source_ips + related.ip; whether these should also populate host.ip should be confirmed against sample data — these are the IPs of source hosts involved in the detection, but the right ECS target depends on the nature of the events
  • source_hosts → crowdstrike.alert.source_hosts

User context

  • user_names / users[].user_name → user.name (first value) + related.user (all values) + crowdstrike.alert.user_names
  • users[].sid → user.id

Alert identification

  • detection_id, display_name, event_ids, pattern_id, vendor_pattern_id → crowdstrike.alert.*
  • display_name → rule.name
  • falcon_host_link → event.url (in addition to existing crowdstrike.alert.falcon_host_link)

Source context

  • source_products[0] → event.provider
  • source_products, source_vendors → crowdstrike.alert.*

Correlation rule fields

  • correlation_rule_id → rule.id + crowdstrike.alert.correlation_rule_id
  • correlation_rule_execution_id, correlation_rule_version_id, correlation_rule_user_id, correlation_rule_user_uuid, correlation_rule_create_case → crowdstrike.alert.*

MITRE ATT&CK

  • mitre_attack → crowdstrike.alert.mitre_attack; when the array is non-empty, populate threat.* fields following the existing EPP pattern in this pipeline — the structure of populated mitre_attack entries should be verified against sample data before implementing

Cleanup

  • Remove the temporary json object at the end of the pipeline and drop null/empty values, consistent with the pattern used in other alert pipelines (e.g. Microsoft Sentinel alert pipeline)

Acceptance criteria

  • event.severity, event.start, event.end, and event.duration are populated
  • host.name, user.name, and rule.name are populated for NG-SIEM events
  • related.hosts, related.user, and related.ip are populated
  • rule.id is populated from correlation_rule_id
  • All correlation_rule_* fields land under crowdstrike.alert.*
  • json.* is removed at the end of pipeline processing
  • Pipeline test added with expected output validated against sample events
  • Field definitions added to fields.yml for any new crowdstrike.alert.* fields introduced
  • Existing EPP alert pipeline tests continue to pass

Activity

  1. jamiehynds
    added
    Integration:crowdstrikeCrowdStrike
    enhancementNew feature or request
    Team:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]
    on May 18, 2026

  2. infra-vault-gh-plugin-prod commented on May 18, 2026

    @infra-vault-gh-plugin-prod
    infra-vault-gh-plugin-prodBot
    on May 18, 2026

    Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

  3. github-actions commented on May 18, 2026

    @github-actions
    github-actionsBot
    on May 18, 2026 – with GitHub Actions
    Contributor

    tl;dr: the alert pipeline appears partially prepared but not explicitly validated/mapped for NG-SIEM; the safest path is to add NG-SIEM-specific normalization in the ingest pipeline and add NG-SIEM fixtures/tests so support is verifiable.

    Recommendation

    Implement NG-SIEM support in packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml by mapping NG-SIEM classification fields before they are removed, and add dedicated NG-SIEM test fixtures in alert pipeline tests. This addresses both behavior and evidence gaps.

    Findings
    • packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs:43 currently pulls alerts from only /alerts/combined/alerts/v1.
    • packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml:51-58 sets ECS event.category/event.type only for process/start conditions.
    • packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml:2248-2265,2281 removes several generic alert classification fields, including:
      • crowdstrike.alert.event_category
      • crowdstrike.alert.event_type
      • crowdstrike.alert.event_name
      • crowdstrike.alert.event_source
      • crowdstrike.alert.service
    • Alert pipeline expected fixtures currently include products epp, idp, overwatch, fcs, but not ngsiem:
      • packages/crowdstrike/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json:186,404,602,929,1269,1447,1522,1603,1686,1891,2066
    • ngsiem is present in benchmark config enum, indicating intended domain support:
      • packages/crowdstrike/_dev/benchmark/system/alert-benchmark/config.yml:58
    • Relevant prior changes:
      • PR #15291 migrated alert collection to /alerts/combined/alerts/v1.
      • PR #18158 intentionally kept unknown/new fields under crowdstrike.alert while explicitly removing selected unmapped fields.

    Note: I could not read issue bodies (including #19048) due integrity restrictions in this run, so this recommendation is based on code and PR evidence.

    Verification

    I ran repository searches to verify current state:

    $ rg -n "alerts/combined/alerts/v1|alerts/queries/alerts|alerts/entities/alerts" packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs
packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs:43: ... "/alerts/combined/alerts/v1?"
packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs:100: ... "/alerts/combined/alerts/v1:"

$ rg -n "remove_intentionally_unmapped_fields|event_category|event_type|event_name|event_source|service" packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml
...:2248: tag: remove_intentionally_unmapped_fields
...:2261: - crowdstrike.alert.event_category
...:2263: - crowdstrike.alert.event_name
...:2264: - crowdstrike.alert.event_source
...:2265: - crowdstrike.alert.event_type
...:2281: - crowdstrike.alert.service

$ rg -n "\"product\"\s*:\s*\"(ngsiem|epp|idp|overwatch|fcs)\"" packages/crowdstrike/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json
...:186: "product": "epp"
...:404: "product": "idp"
...:1447: "product": "overwatch"
...:2066: "product": "fcs"
(no ngsiem matches)

$ rg -n "ngsiem" packages/crowdstrike/_dev/deploy/docker/files/config-alert.yml
No matches found.

$ rg -n "ngsiem" packages/crowdstrike/_dev/benchmark/system/alert-benchmark/config.yml
...:58: - ngsiem
    Detailed Action Plan
    1. Add an NG-SIEM fixture event to packages/crowdstrike/data_stream/alert/_dev/test/pipeline/test-alert.log with representative event_category/event_type/event_name/event_source/service fields.
    2. Update packages/crowdstrike/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json to assert ECS outcomes for that fixture (event.category, event.type, event.action, and preserved vendor fields as applicable).
    3. In packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml, add conditional mappings for NG-SIEM classification fields before remove_intentionally_unmapped_fields.
    4. Revisit the remove list in default.yml (block at remove_intentionally_unmapped_fields) so needed NG-SIEM fields are either mapped then removed (duplicate cleanup) or retained under crowdstrike.alert when appropriate.
    5. Extend system mock responses in packages/crowdstrike/_dev/deploy/docker/files/config-alert.yml with one NG-SIEM example so local/system validation includes this path.
    6. Add changelog entry in packages/crowdstrike/changelog.yml describing NG-SIEM alert pipeline support and test coverage addition.

    If NG-SIEM payload retrieval differs from current combined endpoint behavior, consider a follow-up to add endpoint-mode branching in packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs (similar to host stream patterns).

    Related Items
    Type Link Relevance
    Issue #19048 Target triage item (body unavailable in this run)
    PR #15291 Alert stream migrated to /alerts/combined/alerts/v1
    PR #18158 Alert mapping simplification; unknown fields strategy and explicit removals
    File packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs:43 Current alert endpoint selection
    File packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml:2248-2281 Explicitly removed alert classification fields
    File packages/crowdstrike/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json No NG-SIEM product fixtures today
    File packages/crowdstrike/_dev/benchmark/system/alert-benchmark/config.yml:58 ngsiem appears in benchmark enum

    Note

    🔒 Integrity filter blocked 3 items

    The following items were blocked because they don't meet the GitHub integrity level.

    • #19048 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #19048 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #14135 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

    To allow these resources, lower min-integrity in your GitHub frontmatter:

    tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

    What is this? | From workflow: Issue Triage

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  4. narph
    added
    Team:SDE-CrestCrest developers on the Security Integrations team [elastic/sit-crest-contractors]
    on May 19, 2026
  5. kcreddy
    assigned
    navnit-elastic
    on May 20, 2026
  6. kcreddy
    added the
    Enhancement
    issue type on Jun 3, 2026
  7. navnit-elastic
    closed this as completedin #19494on Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Integration:crowdstrikeCrowdStrikeTeam:SDE-CrestCrest developers on the Security Integrations team [elastic/sit-crest-contractors]Team:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]enhancementNew feature or request

Type

Enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions