ssi_all: use triple-brace templating in complex templates - Part 2 by kcreddy · Pull Request #17623 · elastic/integrations

kcreddy · 2026-03-02T16:29:14Z

Proposed commit message

ssi_all: use triple-brace Mustache templating in ingest pipelines

Fix remaining double-brace Mustache template variables in ingest
pipelines for packages owned by elastic/security-service-integrations.

This is a follow-up to #11314 which addressed most packages but missed
some. Affected packages: admin_by_request_epm,
aws_securityhub, beyondinsight_password_safe, bitsight, cloudflare,
cloudflare_logpush, google_workspace, m365_defender,
microsoft_exchange_online_message_trace, rapid7_insightvm,
sentinel_one, tenable_ot_security, ti_cif3.

Relates: #7641

[git-generate]
for f in $(
	(
		for p in $(
			for f in packages/*/manifest.yml; do
				if yq -e 'select(.owner.github == "elastic/security-service-integrations")' "$f" >/dev/null 2>&1; then
					dirname "$f"
				fi
			done
		); do
			rg -l '(^|[^{])\{\{[^{][ .a-zA-Z0-9_]*[^}]\}\}($|[^}])' -g '*.yml' "$p" || true
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link https://github.com/elastic/integrations/pull/17623
	)>/dev/null 2>&1
done

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

elasticmachine · 2026-03-02T16:44:43Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

elastic-vault-github-plugin-prod · 2026-03-02T20:19:27Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6

When I squash the four commits into one and give it the commit message that's proposed (with the PR number added), we don't get exactly what is here. There are some inconsequential YAML and whitespace changes, but there is also a reversion of the ti_cif3 changes. Can you check that?

In cases where git-generate is used, I'd recommend keeping the PR as a single commit so that the state in the PR is essentially as it will be once it's merged.

kcreddy · 2026-03-03T12:18:11Z

When I squash the four commits into one and give it the commit message that's proposed (with the PR number added), we don't get exactly what is here. There are some inconsequential YAML and whitespace changes, but there is also a reversion of the ti_cif3 changes. Can you check that?

In cases where git-generate is used, I'd recommend keeping the PR as a single commit so that the state in the PR is essentially as it will be once it's merged.

@efd6, the problem was rg wasn't identifying ti_cif3 file because the original pattern uses [^}] after }} which requires a character to exist, missing cases where }} is at the end of a line. I updated the rg regex to match with perl regex to support lookarounds. Now ti_cif3 is included.
I also retained the whitespace and yaml formatting changes that are being done by elastic-package test pipeline and elastic-package changelog add and squashed into single commit.

efd6

Please use

ssi_all: use triple-brace Mustache templating in ingest pipelines

Fix remaining double-brace Mustache template variables in ingest
pipelines for packages owned by elastic/security-service-integrations.

This is a follow-up to #11314 which addressed most packages but missed
some. Affected packages: admin_by_request_epm,
aws_securityhub, beyondinsight_password_safe, bitsight, cloudflare,
cloudflare_logpush, google_workspace, m365_defender,
microsoft_exchange_online_message_trace, rapid7_insightvm,
sentinel_one, tenable_ot_security, ti_cif3.

Relates: #7641

[git-generate]
for f in $(
	(
		for p in $(
			for f in packages/*/manifest.yml; do
				if yq -e 'select(.owner.github == "elastic/security-service-integrations")' "$f" >/dev/null 2>&1; then
					dirname "$f"
				fi
			done
		); do
			rg -l '(^|[^{])\{\{[^{][ .a-zA-Z0-9_]*[^}]\}\}($|[^}])' -g '*.yml' "$p" || true
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link https://github.com/elastic/integrations/pull/17623
	)>/dev/null 2>&1
done

This does not depend on PCRE and fixes the missing issue number. You can rebase this onto main and use the -conflict flag in git generate when you get to conflicts during the rebase.

Fix remaining double-brace Mustache template variables in ingest pipelines for packages owned by elastic/security-service-integrations. This is a follow-up to elastic#11314 which addressed most packages but missed some. Affected packages: admin_by_request_epm, aws_securityhub, beyondinsight_password_safe, bitsight, cloudflare, cloudflare_logpush, google_workspace, m365_defender, microsoft_exchange_online_message_trace, rapid7_insightvm, sentinel_one, tenable_ot_security, ti_cif3. Relates: elastic#7641 [git-generate] for f in $( ( for p in $( for f in packages/*/manifest.yml; do if yq -e 'select(.owner.github == "elastic/security-service-integrations")' "$f" >/dev/null 2>&1; then dirname "$f" fi done ); do rg -l '(^|[^{])\{\{[^{][ .a-zA-Z0-9_]*[^}]\}\}($|[^}])' -g '*.yml' "$p" || true done )|grep "elasticsearch/ingest_pipeline"|sort|uniq ); do perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f done for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do ( cd $p elastic-package test pipeline -g elastic-package changelog add \ --description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \ --type bugfix \ --next patch \ --link elastic#17623 )>/dev/null 2>&1 done

elasticmachine · 2026-03-04T09:14:30Z

💚 Build Succeeded

Buildkite Build
Commit: b2133eb

History

💚 Build #39060 succeeded 6d59d01
💔 Build #39058 failed f71a073
💚 Build #39010 succeeded fd27899
💔 Build #39009 failed c90bc2d

cc @kcreddy

elastic-vault-github-plugin-prod · 2026-03-05T03:44:41Z

Package admin_by_request_epm - 1.1.2 containing this change is available at https://epr.elastic.co/package/admin_by_request_epm/1.1.2/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:42Z

Package aws_securityhub - 0.2.1 containing this change is available at https://epr.elastic.co/package/aws_securityhub/0.2.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:43Z

Package beyondinsight_password_safe - 0.12.3 containing this change is available at https://epr.elastic.co/package/beyondinsight_password_safe/0.12.3/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:43Z

Package bitsight - 0.1.1 containing this change is available at https://epr.elastic.co/package/bitsight/0.1.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:44Z

Package cloudflare - 2.32.1 containing this change is available at https://epr.elastic.co/package/cloudflare/2.32.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:45Z

Package cloudflare_logpush - 1.43.3 containing this change is available at https://epr.elastic.co/package/cloudflare_logpush/1.43.3/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:46Z

Package google_workspace - 3.3.1 containing this change is available at https://epr.elastic.co/package/google_workspace/3.3.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:47Z

Package m365_defender - 5.10.1 containing this change is available at https://epr.elastic.co/package/m365_defender/5.10.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:48Z

Package microsoft_exchange_online_message_trace - 2.0.1 containing this change is available at https://epr.elastic.co/package/microsoft_exchange_online_message_trace/2.0.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:50Z

Package rapid7_insightvm - 2.6.1 containing this change is available at https://epr.elastic.co/package/rapid7_insightvm/2.6.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:51Z

Package sentinel_one - 2.4.1 containing this change is available at https://epr.elastic.co/package/sentinel_one/2.4.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:52Z

Package tenable_ot_security - 1.0.1 containing this change is available at https://epr.elastic.co/package/tenable_ot_security/1.0.1/

elastic-vault-github-plugin-prod · 2026-03-05T03:44:53Z

Package ti_cif3 - 1.18.2 containing this change is available at https://epr.elastic.co/package/ti_cif3/1.18.2/

kcreddy self-assigned this Mar 2, 2026

kcreddy marked this pull request as ready for review March 2, 2026 16:40

kcreddy requested a review from a team as a code owner March 2, 2026 16:40

kcreddy added bugfix Pull request that fixes a bug issue Category: Integration quality Category: Quality used for SI planning labels Mar 2, 2026

efd6 reviewed Mar 3, 2026

View reviewed changes

kcreddy force-pushed the triple-braces branch 2 times, most recently from f71a073 to 6d59d01 Compare March 3, 2026 12:14

kcreddy requested a review from efd6 March 3, 2026 12:20

efd6 reviewed Mar 3, 2026

View reviewed changes

kcreddy force-pushed the triple-braces branch from 6d59d01 to b2133eb Compare March 4, 2026 05:51

kcreddy requested a review from efd6 March 4, 2026 09:14

efd6 approved these changes Mar 4, 2026

View reviewed changes

kcreddy merged commit 6b441ed into elastic:main Mar 5, 2026
9 checks passed

github-actions Bot mentioned this pull request Mar 30, 2026

microsoft_exchange_online_message_trace: endpoint returns broken skiptoken causing infinite 400 loop #18134

Closed

github-actions Bot mentioned this pull request Apr 16, 2026

[Elasticsearch]: Authentication fails when password contains YAML special characters (e.g. double quotes) #18434

Closed

github-actions Bot mentioned this pull request May 4, 2026

ssi_all: cel integrations do not properly guard against null #18788

Open

coderabbitai Bot mentioned this pull request Aug 25, 2026

Kubernetes test cases + skip_initial_seconds correctness knob VirtualMetric/PipeBench#57

Closed

Conversation

kcreddy commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Related issues

Uh oh!

elasticmachine commented Mar 2, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

kcreddy commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 4, 2026

💚 Build Succeeded

History

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kcreddy commented Mar 2, 2026 •

edited

Loading

elastic-vault-github-plugin-prod Bot commented Mar 2, 2026 •

edited

Loading

kcreddy commented Mar 3, 2026 •

edited

Loading