Repository navigation
ssi_all: cel integrations do not properly guard against null #18788
Description
Activity
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
tl;dr: The strongest evidence points to a null-guard gap in the Proofpoint Essentials CEL stream for
/v2/siem/all(packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs), where response fields are dereferenced as arrays/timestamps without optional/null protection.Recommendation
Update the
proofpoint_essentialsthreat CEL program to treat nullable API fields as optional before mapping/parsing, then add/extend stream-level coverage fornullresponse members. This is the most likely place matching the issue title (ssi_allappears to correspond to/v2/siem/all).Findings
-
Unguarded nullable field access in CEL stream
packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs:75-78maps directly over:body.clicksBlockedbody.clicksPermittedbody.messagesBlockedbody.messagesDelivered
packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs:82,84parsesbody.queryEndTimedirectly.
-
No local
ssi_allsymbol in repo; endpoint match is/v2/siem/allgit grep -i "ssi_all" -- packagesreturned no matches.packages/proofpoint_essentials/docs/README.md:10,14explicitly documents/v2/siem/allas the threat source endpoint.
-
Established safe pattern elsewhere in CEL streams
- Example guarded mapping pattern:
packages/microsoft_sentinel/data_stream/incident/agent/stream/cel.yml.hbs:46body.?value.orValue([]).map(...)
- This pattern avoids null dereference on optional arrays.
- Example guarded mapping pattern:
-
Parallel investigation outcome (Pick Three, Keep One)
- 3 sub-agents ran with distinct angles.
- Selected winner: Proofpoint
/v2/siem/allstream finding above (best evidence + issue-title alignment). - Other candidates were lower-confidence or broader (
noop/ unrelated stream).
Verification
I validated with local commands:
$ git --no-pager grep -n -i "ssi_all" -- packages # (no matches) $ nl -ba packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs | sed -n '67,86p' 67 ... "/v2/siem/all?..." 75 body.clicksBlocked.map(...) 76 body.clicksPermitted.map(...) 77 body.messagesBlocked.map(...) 78 body.messagesDelivered.map(...) 82 ... body.queryEndTime.parse_time(...) 84 ... body.queryEndTime.parse_time(...) $ nl -ba packages/microsoft_sentinel/data_stream/incident/agent/stream/cel.yml.hbs | sed -n '45,48p' 46 "events": body.?value.orValue([]).map(e, {
Detailed Action Plan
-
Harden nullable lists in Proofpoint threat stream
- File:
packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs - Replace direct list maps (
:75-78) with guarded defaults, e.g.body.?clicksBlocked.orValue([]).map(...)(and same for the other 3 lists).
- File:
-
Harden nullable timestamp parsing
- Same file (
:82,84) - Guard
queryEndTimebeforeparse_time, with fallback strategy:- fallback cursor to
time_range.end_time, or - skip
want_morecontinuation when timestamp is absent and emit structured error event.
- fallback cursor to
- Same file (
-
Add regression coverage for null fields
- Add/extend stream test input(s) for HTTP 200 response with one or more null members (e.g.,
clicksBlocked: null,queryEndTime: null). - Assert stream does not error and either emits zero events or the non-null event sets only.
- Add/extend stream test input(s) for HTTP 200 response with one or more null members (e.g.,
-
Sanity-check docs/examples if needed
- If behavior changes in cursor semantics, update docs/comments where
/v2/siem/allpagination behavior is described.
- If behavior changes in cursor semantics, update docs/comments where
Related Items
Type Link/Path Relevance Issue #18788Triage target File packages/proofpoint_essentials/data_stream/threat/agent/stream/cel.yml.hbs:75-78,82,84Unguarded nullable dereferences File packages/proofpoint_essentials/docs/README.md:10,14/v2/siem/allendpoint mappingFile packages/microsoft_sentinel/data_stream/incident/agent/stream/cel.yml.hbs:46Known-good null-safe CEL pattern Issues/PRs search keyword searches in repo No directly matching prior issue/PR found via available queries Note
🔒 Integrity filter blocked 7 items
The following items were blocked because they don't meet the GitHub integrity level.
- #18788
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #18788
search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #17623
search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #11284
search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #11314
search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #12047
search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #12046
search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower
min-integrityin your GitHub frontmatter:tools: github: min-integrity: approved # merged | approved | unapproved | none
What is this? | From workflow: Issue Triage
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
Reacted by Dan Kortschak-
- added 4 commits that reference this issue
on May 22, 2026
The data streams below all have the pattern
has(x.f) && size(x.f) != 0or similar. This is used to guard against empty arrays. However, it does not guard againstnullin place of an empty array, and idiom that is reasonably common (an example of this caused a runtime failure in theaws.configdata stream (ref).The pattern is not always unsafe; it may be that the program sets the field to an array and so there is static proof that the test is always successful, or the API might document that the value is always an array. It might though just be simpler to always apply the fix (
has(x.f) && x.f != null && size(x.f) != 0).