[mimecast]: Pipeline error for avlog events in siem_logs dataset #19032

Integration Name

Mimecast [packages/mimecast]

Dataset Name

mimecast.siem_logs

Integration Version

3.3.2

Agent Version

9.3.4

Agent Output Type

elasticsearch

Elasticsearch Version

9.3.4

OS Version and Architecture

Ubuntu 24.04 LTS (x86-64)

Software/API Version

No response

Error Message

field [source.domain] already exists

Event Original

{"accountId":"redacted","aggregateId":"redacted","emailSize":"1234","fileExtension":"zip","fileName":"eicar.zip","md5":"40be8f71c7eb6fc53bba6e2edc1c5af2","messageId":"\u003cNDBiZThmNzFjN2ViNmZjNTNiYmE2ZTJlZGMxYzVhZjI@sender.example.com\u003e","processingId":"redacted","recipients":"bob@recipient.example.org","route":"inbound","senderDomain":"sender.example.com","senderDomainInternal":"false","senderEnvelope":"alice@sender.example.com","senderIp":"198.51.100.10","sha1":"22fd3d86d8ec05c28d42fab2b765ba4e89147552","sha256":"f99da86bb6bd89416ff53fed876f48330fd8b9505a456d9968ae1321e331babb","subject":"Test","subtype":null,"timestamp":1778842800000,"type":"av","virusFound":"Malware detected by AV Scan policy: Eicar-Test-Signature"}

What did you do?

The integration is configured to collect SIEM logs, including the av log type, using the Mimecast v2 API.

What did you see?

There are pipeline errors when ingesting avlog events because it is trying to set source.domain when it already exists. The pipeline renames mimecast.senderDomainInternal to source.domain and later tries to rename mimecast.senderDomain to source.domain.

What did you expect to see?

The source.domain field should be set to the value of mimecast.senderDomain. The mimecast.senderDomainInternal field is a boolean indicating whether the sender domain is internal, so this should be kept as a separate field and not renamed to source.domain.

Anything else?

No response