Skip to content

backport: deploy backport-branch automation workflows to active backport branches #19262

Closed
Closed
backport: deploy backport-branch automation workflows to active backport branches#19262
Assignees
mrodm
Labels
Team:EcosystemPackages Ecosystem team [elastic/ecosystem]

Description

@mrodm
mrodm
opened on May 28, 2026

Summary

Several workflows trigger on events scoped to backport-* branches — sync-backport-changelog.yml (#19215) on push, post-backport-branch.yml (#19214) on pull_request, and the owner-sync CI check (#19686) on pull_request — and GitHub Actions resolves both event types from the workflow files present on the branches involved in the event, not from main:

  • push workflows execute from the workflow file on the branch being pushed to.
  • pull_request (non-pull_request_target) workflows execute from a merge of the PR's head and base branches.

In both cases, if the workflow file is absent from the backport branch, GitHub never schedules it — the automation is silently skipped for every existing branch.

This issue covers the one-time migration to add these workflows (and any scripts they call) to all currently active branches.

This issue is a follow-up to #19215, #19214, and #19686 — existing branches will not benefit from this automation until the migration is run, but each of those issues can ship and be useful for all newly created branches immediately.

Background

The existing backport branches were created from commits that predate the backport automation work. The backport_branch.sh script already copies the required files when creating new branches, so new branches are covered from day one. The gap is the existing active branches, which need a one-time migration after #19215, #19214, and #19686 ship.

There is no workaround via workflow_run or workflow_dispatch — both push and pull_request events on a backport branch will only fire workflows that already exist on that branch (or, for pull_request, on the head branch cut from it).

Deliverables

One-time migration script

A script (or documented manual procedure) that, for each active branch in .backports.yml:

  1. Checks out the backport branch
  2. Copies .github/workflows/sync-backport-changelog.yml, .github/workflows/post-backport-branch.yml, the owner-sync CI check workflow introduced in Sync package owners between backport branches and main #19686, and any scripts they call (resolved at implementation time from each issue's deliverables) onto the branch

    Consider reusing the same approach backport_branch.sh already uses to copy required files from main when creating new branches (git checkout main -- <path> per file/directory) instead of building a separate ad hoc copy mechanism for this migration script.

  3. Opens a PR against the backport branch with the label automation

The script can be run once by the ecosystem team after #19215, #19214, and #19686 ship. A migration PR per branch is preferred over a force-push so that the addition goes through CI and review.

Note: the exact set of files to copy is determined by #19215's, #19214's, and #19686's final implementations. Coordinate with those issues before running the migration.

Considerations

Acceptance criteria

  • All currently active branches listed in .backports.yml have sync-backport-changelog.yml, post-backport-branch.yml, and the owner-sync CI check (and their dependencies) present
  • Pushing to any active backport branch after migration triggers sync-backport-changelog.yml correctly
  • Opening an auto-backport PR against any active branch after migration correctly triggers post-backport-branch.yml
  • All currently active branches enforce the owner-sync CI check from Sync package owners between backport branches and main #19686 on PRs targeting them
  • Migration PRs are labelled automation and do not trigger auto-backport.yml on merge

Activity

  1. mrodm
    added a parent issue on May 28, 2026
  2. mrodm
    marked this as blocked by 2 issues on May 28, 2026
  3. mrodm
    added
    Team:EcosystemPackages Ecosystem team [elastic/ecosystem]
    on May 28, 2026

  4. infra-vault-gh-plugin-prod commented on May 28, 2026

    @infra-vault-gh-plugin-prod
    infra-vault-gh-plugin-prodbot
    on May 28, 2026

    Pinging @elastic/ecosystem (Team:Ecosystem)

  5. github-actions commented on May 28, 2026

    @github-actions
    github-actionsbot
    on May 28, 2026 – with GitHub Actions
    Contributor

    tl;dr: This should be sequenced as a post-#19215 migration, but it is not executable yet from current main because .backports.yml is missing (no authoritative active-branch list) and the sync workflow file is not present in this checkout.

    Recommendation

    Proceed with this issue, but gate execution on two prerequisites:

    1. #19215 is merged and the final sync-backport-changelog file set is known.
    2. An authoritative active-branch inventory exists (the issue currently points to .backports.yml, which is not in current main).

    Then run a one-time per-branch PR migration (not force-push), label each PR automation, and explicitly trigger CI for workflow-only changes.

    Findings
    • No backport/changelog workflow exists in current checkout (so there is nothing to deploy yet from this branch state).
      • Command: ls .github/workflows | grep -E 'backport|changelog' || true (no output)
    • .backports.yml does not exist in current checkout, so the acceptance criterion “branches listed in .backports.yml” cannot be executed yet.
      • Command: test -f .backports.yml && echo present || echo missing → missing
    • There are many existing backport branches that would need targeting once inventory is defined.
      • Command: git ls-remote --heads origin 'backport-*' | wc -l → 58
    • New branches are already covered by existing branch-creation flow, because the script copies workflows from main when creating backport branches.
      • .buildkite/scripts/backport_branch.sh:206-211
    • Current docs still require manual changelog sync to main, which aligns with why #19215/#19262 are needed.
      • docs/extend/developer-workflow-support-old-package.md:117-121
    • Migration PRs touching only .github/workflows/ can skip default Buildkite PR CI unless manually triggered.
      • .buildkite/pull-requests.json:16,26,38
    Verification 
    $ ls .github/workflows | grep -E "backport|changelog" || true

$ test -f .backports.yml && echo present || echo missing
missing

$ git ls-remote --heads origin "backport-*" | wc -l
58

$ grep -nE "skip_ci_on_only_changed|always_require_ci_on_changed|\^\.github/workflows/" .buildkite/pull-requests.json
16:      "skip_ci_on_only_changed": [
26:        "^.github/workflows/",
38:      "always_require_ci_on_changed": []
    Detailed Action Plan

    1. Finalize dependencies from #19215

      • Confirm exact migration payload (at minimum .github/workflows/sync-backport-changelog.yml, plus any invoked scripts/actions).
      • Source: #19262 explicitly defers exact file set to #19215 final implementation.

    2. Establish active branch source of truth

      • If .backports.yml is intended, land/populate it first.
      • If another source is now canonical, update #19262 acceptance criteria to reference it explicitly.

    3. Generate one PR per active backport branch

      • For each active branch, create a migration branch based on that backport branch.
      • Copy the approved file set from main.
      • Open PR targeting the backport branch.
      • Apply automation label at PR creation.

    4. Prevent silent CI gaps for workflow-only PRs

      • Because .github/workflows/-only changes can skip default Buildkite PR CI (.buildkite/pull-requests.json:26), run the required PR checks explicitly (e.g., manual trigger policy used by the team).

    5. Roll out and verify

      • Merge migration PRs.
      • Push a harmless test commit to at least one migrated backport branch to confirm sync-backport-changelog.yml is picked up from branch context.

    6. Future-proof file propagation

      • Add/maintain a small manifest of “files required on every backport branch” and use it both for branch creation and migration reruns.
    Related Items
    Type Link Relevance
    Issue #19262 This migration issue
    Issue #19215 Defines sync-backport-changelog workflow and dependency payload
    Issue #19214 Adjacent auto-backport/post-backport workflows; automation label behavior
    Issue #19016 Umbrella backport automation effort
    PR #19104 Landed helper/docs groundwork for backport flow
    File .buildkite/scripts/backport_branch.sh:90-106 Existing package name→path resolution helper
    File .buildkite/scripts/backport_branch.sh:206-211 New backport branches copy .github/workflows from main
    File .buildkite/pull-requests.json:16,26,38 Workflow-only changes can skip default Buildkite PR CI
    File docs/extend/developer-workflow-support-old-package.md:117-121 Manual changelog sync step still documented

    Note

    🔒 Integrity filter blocked 5 items

    The following items were blocked because they don't meet the GitHub integrity level.

    • #19024 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #17641 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #14306 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #10895 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
    • #19171 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

    To allow these resources, lower min-integrity in your GitHub frontmatter:

    tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

    What is this? | From workflow: Issue Triage

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  6. juliaElastic commented on Jun 24, 2026

    @juliaElastic
    juliaElastic
    on Jun 24, 2026
    Contributor

    @mrodm @jsoriano The blockers are both closed, is this issue ready to work on in next sprint?

  7. mrodm commented on Jun 26, 2026

    @mrodm
    mrodm
    on Jun 26, 2026
    CollaboratorAuthor

    @mrodm @jsoriano The blockers are both closed, is this issue ready to work on in next sprint?

    @juliaElastic Yes, this could be done as part of the next sprint. We will need to check if all the CI scripts should be updated, maybe there are some backport branches that could have the CI pipeline broken now.

    It could be worth also picking up #19213 in the same sprint; it's unblocked too.

  8. mrodm commented on Jul 3, 2026

    @mrodm
    mrodm
    on Jul 3, 2026
    CollaboratorAuthor

    It would be better to wait for this PR to be merged #19930 before starting with this issue

    It also needs to be checked if it is needed to sync other changes from other backport workflows or scripts and mage targets used in the workflows. If so, it could be done at the same time.

  9. mrodm commented on Jul 6, 2026

    @mrodm
    mrodm
    on Jul 6, 2026
    CollaboratorAuthor

    post-backport-branch.yml workflow (to be introduced in #19214) has the same failure mode described here, just via a different trigger. It uses pull_request: opened targeting backport-*, and non-pull_request_target events resolve the workflow definition from a merge of the PR's head and base branches — not from main. Since the head branch (auto-backport/<package>-<major.minor>-<sha>) is always cut directly from the backport branch, and neither branch touches .github/workflows/, the file effectively has to already exist on the backport branch for GitHub to schedule the workflow at all. Existing active branches predate #19214, so they'd hit the same silent no-op this issue was written to fix for sync-backport-changelog.yml.

    Same happens once this issue #19686 would be fixed. It would require the scripts and mage targets to be present in the backport branches that were already created.

    Given that, I think it would be better to update this issue related to sync scripts, workflows and mage targets instead of filing a separate follow-up issue later. If agreed, I'll broaden the title to something like backport: deploy backport-branch automation workflows to active backport branches and update the issue description to include post-backport-branch.yml in the migration scope. cc @elastic/ecosystem @bturquet

    Doing so, this issue would be blocked by #19214 and #19686 .

  10. mrodm
    changed the title [-]backport: deploy sync-backport-changelog workflow to active backport branches[/-] [+]backport: deploy backport-branch automation workflows to active backport branches[/+] on Jul 8, 2026
  11. mrodm
    marked this as blocked by 2 issues on Jul 8, 2026

  12. mrodm commented on Jul 8, 2026

    @mrodm
    mrodm
    on Jul 8, 2026
    CollaboratorAuthor

    Talked offline with @teresaromero

    Updated the title and description to describe that it should also be taken into account the changes in #19214 and #19686 for this sync for the backport branches that are already created. And mark it as blocked.

    cc @bturquet

  13. mrodm
    self-assigned this
    on Jul 30, 2026

  14. mrodm commented on Jul 30, 2026

    @mrodm
    mrodm
    on Jul 30, 2026
    CollaboratorAuthor

    Active backport branches

    Generated from .backports.yml on 2026-07-30. Includes branches where archived: false and maintained_until is null or not yet passed.

    aws

    cloud_asset_inventory

    cloud_security_posture

    crowdstrike

    elastic_agent

    gcp

    google_workspace

    kubernetes

    security_detection_engine

    sql

    synthetics

    • backport-synthetics-1.0 [synthetics-1.0] Sync CI configuration with main branch #21171
      • Required to apply some specific changes due to its using an old elastic-package version and it does not recognize some parameters
        • --coverage-format parameter
        • restore the elastic-package stack shellinit command
        • use docker-compose instead of docker compose, and version v2.17.2
        • fix script to detect whether or not stack is running
        • fix codeowners (data streams) for aws, gcp and system packages (copied from main).

    tenable_io

    ti_abusech

    wiz

    Skipped: backport-apm-8.15, backport-aws-7.15.0, backport-elastic_package_registry-0.2, and all archived security_detection_engine branches (8.6 through 9.1 except 8.19).

  15. mrodm commented on Aug 5, 2026

    @mrodm
    mrodm
    on Aug 5, 2026
    CollaboratorAuthor

    Created PRs to fix some issues found while deploying files into backport branches:

  16. mrodm commented on Aug 5, 2026

    @mrodm
    mrodm
    on Aug 5, 2026
    CollaboratorAuthor

    While syncing the new CI configuration into old backport branches I found several errors caused by mismatch of go versions and failures while running dependencies that were updated.

    Trying to find a solution to keep the backport tool as much separated as possible , so it is easy to copy into old backport branches.

  17. mrodm commented on Aug 5, 2026

    @mrodm
    mrodm
    on Aug 5, 2026
    CollaboratorAuthor

    Trying to find a solution to keep the backport tool as much separated as possible , so it is easy to copy into old backport branches.

    Created this PR #20556 that extracts the code necessary for backport automation into its own CLI binary at cmd/backport with its own go.mod. That will allow to copy that folder safely to other backport branches, and it will not update the go.mod from the root of the working copy.

  18. mrodm
    marked this as blocking on Aug 6, 2026

  19. mrodm commented on Aug 7, 2026

    @mrodm
    mrodm
    on Aug 7, 2026
    CollaboratorAuthor

    To avoid another round of Pull Requests updating backport branches, it would be better to update those backport branches once this issue is closed: #19321

    There is a PR already opened for this #20578

  20. mrodm commented on Aug 7, 2026

    @mrodm
    mrodm
    on Aug 7, 2026
    CollaboratorAuthor

    Related PR to update the checklist comment #20605 to include more information for developers

  21. mrodm commented on Sep 7, 2026

    @mrodm
    mrodm
    on Sep 7, 2026
    CollaboratorAuthor

    Given the current branches that should be updated, there are some of these branches that their packages require to run Terraform for their tests.

    One of the latest changes required to fix in elsatic-package was elastic/elastic-package#3876 . The Terraform Dockerfile required to be updated to fix the package names for google-cloud sdk.

    This change should be backported to the backport branches that reuqire Terraform:

    BRANCH                                             PACKAGE                ELASTIC-PACKAGE        TERRAFORM
-------------------------------------------------- ---------------------- ---------------------- --------
backport-aws-7.1                                   aws                    0.126.1                yes (26 files)
backport-aws-6.x                                   aws                    0.125.0                yes (26 files)
backport-aws-3.17                                  aws                    0.114.0                yes (26 files)
backport-aws-3.13                                  aws                    0.113.0                yes (26 files)
backport-aws-2.30                                  aws                    0.106.0                yes (26 files)
backport-aws-2.25                                  aws                    0.104.0                yes (26 files)
backport-aws-2.24                                  aws                    0.103.0                yes (26 files)
backport-aws-1.51                                  aws                    0.85.0                 yes (4 files)
backport-crowdstrike-1.52                          crowdstrike            0.109.1                yes (3 files)
backport-crowdstrike-1.46                          crowdstrike            0.107.1                yes (3 files)

    All the other backport branches would not be easy to use the latest elastic-package version because they are too old and there could be unexpected changes. I'd try if possible to create new patch versions via backport/hotfixes branches.

    cc @teresaromero @bturquet

    EDIT:
    And it would be needed to add the required labels in the terraform resources too.
    Example for crowdstrike: #20706

  22. mrodm commented on Sep 15, 2026

    @mrodm
    mrodm
    on Sep 15, 2026
    CollaboratorAuthor

    This PR #21243 ensures that the Pull Requests created to keep the changelogs in sync (with the backported versions) have an assignee set.

  23. mrodm commented on Sep 17, 2026

    @mrodm
    mrodm
    on Sep 17, 2026
    CollaboratorAuthor

    Backport branches have been updated with all the CI changes to run the backport tool and workflows.

    Moreover, the fix related to assigning an user to the sync changelog PR has been also backported to all branches #21243

    There is a PR to include the assignment of the user in the Pull Requests created targeting backport branches (auto-backport.yml workflow) #21379
    As this workflow just runs on main, it should not be needed to backport it to all the backport branches.

    Once that last PR #21379 is merged, this issue could be closed.

  24. mrodm commented on Sep 18, 2026

    @mrodm
    mrodm
    on Sep 18, 2026
    CollaboratorAuthor

    Closing this issue, since all backport branches updated, and latest fixes found have also been merged into main.

  25. mrodm
    closed this as completedon Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Team:EcosystemPackages Ecosystem team [elastic/ecosystem]

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions