Skip to content

Commit 0e8d953

Browse files
denikIsaac
denik
and
Isaac
committed
direct: downgrade a planned action when a resolved reference leaves it unchanged
A reference to a resource being recreated cannot be resolved at plan time (a recreate does not preserve the id), so the planner diffs the dependent field against the literal "${...}" placeholder and inflates the dependent's action -- a recreate for an immutable/id field, an update for a mutable one -- even when the referenced value does not actually change. At apply, references are resolved in dependency order before the resource is deployed. Re-check the plan's own changes there and drop any that were a phantom of an unresolved reference: a local edit (New != Old, where New was the placeholder) whose field now resolves back to Old. Recompute the action over what remains and downgrade it -- never upgrade. If nothing remains, the resource is skipped instead of recreated/updated. Working from the plan's own changes rather than re-diffing from scratch keeps changes a struct diff would not surface (a cluster's libraries) and remote-only drift (New == Old) intact; child resources (permissions, grants) are excluded because their diff is specialized. The recreate-reference acceptance tests now show the direct engine issuing no spurious request. Co-authored-by: Isaac <no-reply@databricks.com>
1 parent 3b34033 commit 0e8d953

13 files changed

Lines changed: 91 additions & 77 deletions

File tree

‎.nextchanges/bundles/fix-downgrade-spurious-action-on-recreated-ref.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* Direct engine: do not recreate or update a resource that references a resource being recreated when the referenced value is unchanged; the planned action is downgraded once the reference resolves at deploy. ([#6914](https://github.com/databricks/cli/pull/6914))

‎acceptance/bundle/resources/postgres_databases/recreate_ref_spurious_update/databricks.yml.tmpl‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,13 @@ sync:
1515
# field in the PATCH update_mask, so it is delete + create -- see postgres_roles/
1616
# recreate-postgres-role). role_id is unchanged, so my_role.name (projects/.../roles/app-owner)
1717
# is stable. The direct engine cannot resolve ${my_role.name} while the role is recreated, so
18-
# my_database.role diffs the saved name against the "${...}" placeholder and plans an
19-
# unnecessary update.
18+
# my_database.role diffs the saved name against the "${...}" placeholder and the plan shows an
19+
# unnecessary update. At deploy, though, the direct engine downgrades it to a no-op -- it
20+
# re-diffs once ${my_role.name} resolves and sees it unchanged -- so my_database is left alone.
2021
#
2122
# This test runs the direct engine only. For context, the terraform engine (now being removed)
22-
# left my_database alone: against the lenient testserver it updates the role in place, so the
23-
# name reference resolved at plan and never diffed.
23+
# also left my_database alone, by a different route: against the lenient testserver it updates
24+
# the role in place, so the name reference resolved at plan and never diffed.
2425
resources:
2526
postgres_projects:
2627
my_project:

‎acceptance/bundle/resources/postgres_databases/recreate_ref_spurious_update/out.requests.direct.txt‎

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,6 @@
22
"method": "DELETE",
33
"path": "/api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]/branches/main/roles/app-owner"
44
}
5-
{
6-
"method": "PATCH",
7-
"path": "/api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]/branches/main/databases/my-db",
8-
"q": {
9-
"update_mask": "spec.role"
10-
},
11-
"body": {
12-
"spec": {
13-
"postgres_database": "mydb",
14-
"role": "projects/test-pg-proj-[UNIQUE_NAME]/branches/main/roles/app-owner"
15-
}
16-
}
17-
}
185
{
196
"method": "POST",
207
"path": "/api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]/branches/main/roles",

‎acceptance/bundle/resources/postgres_databases/recreate_ref_spurious_update/script‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ trace $CLI bundle plan -o json | nostamp | jq '.plan."resources.postgres_databas
2626

2727
# WHAT ACTUALLY HAPPENED: deploy and record the requests (deploy stdout -> LOG.deploy; parallel
2828
# completion order is nondeterministic). out.requests.direct.txt shows the direct engine issues
29-
# a PATCH against my_database -- a spurious update writing back the unchanged role.
29+
# NO request for my_database: it downgraded the planned update to a no-op once ${my_role.name}
30+
# resolved to the unchanged value.
3031
title "Deploy and record the requests (see out.plan.direct.txt, out.requests.direct.txt)"
3132
$CLI bundle deploy --auto-approve &> LOG.deploy
3233
trace print_requests.py --sort '//postgres' > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt

‎acceptance/bundle/resources/postgres_projects/recreate_ref_both_engines/databricks.yml.tmpl‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,15 @@ sync:
1313
#
1414
# pg_version is spec:immutable, so a bump recreates the project. project_id is unchanged, so the
1515
# project name (projects/{project_id}) is stable, and my_catalog.branch -- built from
16-
# ${my_project.name} -- resolves to the same value: the catalog does not actually change, yet
17-
# the direct engine plans and executes its recreate.
16+
# ${my_project.name} -- resolves to the same value: the catalog does not actually change. The
17+
# plan still shows its recreate (the reference cannot be resolved until the project is deployed),
18+
# but at deploy the direct engine downgrades it to a no-op -- see out.requests.direct.txt, which
19+
# has only the project's calls.
1820
#
1921
# This test runs the direct engine only. For context, the terraform engine (now being removed)
20-
# behaved the same here -- it also recreated the catalog, because it cannot cancel a replace
21-
# once the plan commits it to separate destroy/create graph nodes.
22+
# did NOT downgrade this -- it recreated the catalog, because it cannot cancel a replace once the
23+
# plan commits it to separate destroy/create graph nodes. The direct engine's apply-time
24+
# downgrade is strictly more capable here.
2225
resources:
2326
postgres_projects:
2427
my_project:

‎acceptance/bundle/resources/postgres_projects/recreate_ref_both_engines/out.requests.direct.txt‎

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,7 @@
1-
{
2-
"method": "DELETE",
3-
"path": "/api/2.0/postgres/catalogs/mycat"
4-
}
51
{
62
"method": "DELETE",
73
"path": "/api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]"
84
}
9-
{
10-
"method": "POST",
11-
"path": "/api/2.0/postgres/catalogs",
12-
"q": {
13-
"catalog_id": "mycat"
14-
},
15-
"body": {
16-
"spec": {
17-
"branch": "projects/test-pg-proj-[UNIQUE_NAME]/branches/production",
18-
"create_database_if_missing": true,
19-
"postgres_database": "appdb"
20-
}
21-
}
22-
}
235
{
246
"method": "POST",
257
"path": "/api/2.0/postgres/projects",

‎acceptance/bundle/resources/postgres_projects/recreate_ref_both_engines/output.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ recreate postgres_projects.my_project
1717

1818
Plan: 2 to add, 0 to change, 2 to delete, 0 unchanged
1919

20-
=== Deploy the recreate and record the requests (see out.requests.direct.txt)
20+
=== Deploy and record the requests (see out.requests.direct.txt)
2121
>>> print_requests.py --sort //postgres
2222

2323
>>> [CLI] bundle destroy --auto-approve

‎acceptance/bundle/resources/postgres_projects/recreate_ref_both_engines/script‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,11 @@ trace update_file.py databricks.yml "pg_version: 16" "pg_version: 17"
2323
title "Plan"
2424
trace $CLI bundle plan
2525

26-
# WHAT ACTUALLY HAPPENED: the direct engine performs a real delete + create of the catalog.
27-
# Deploy stdout goes to LOG.deploy (parallel completion order is nondeterministic, so it is not
28-
# a golden); the API calls are recorded below. out.requests.direct.txt shows the DELETE + POST
29-
# issued against the catalog with an identical, unchanged spec -- a destructive recreate of a
30-
# resource that did not change.
31-
title "Deploy the recreate and record the requests (see out.requests.direct.txt)"
26+
# WHAT ACTUALLY HAPPENED: the direct engine downgrades the catalog recreate to a no-op -- once
27+
# the project is recreated it re-diffs with ${my_project.name} resolved, sees my_catalog.branch
28+
# unchanged, and skips the catalog -- so out.requests.direct.txt has only the project's DELETE +
29+
# POST. Deploy stdout goes to LOG.deploy (parallel completion order is nondeterministic, so it
30+
# is not a golden).
31+
title "Deploy and record the requests (see out.requests.direct.txt)"
3232
$CLI bundle deploy --auto-approve &> LOG.deploy
3333
trace print_requests.py --sort '//postgres' > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt

‎acceptance/bundle/resources/schemas/recreate_id_ref_spurious_update/databricks.yml.tmpl‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,12 @@ sync:
1616
# as .id (a schema's full_name "catalog.schema").
1717
#
1818
# referent recreates on storage_root (immutable, non-identity), so its full_name / .id
19-
# (main.referent) is unchanged. consumer references ${referent.id} in its mutable comment, so
20-
# the direct engine plans a spurious update of consumer.
19+
# (main.referent) is unchanged. consumer references ${referent.id} in its mutable comment, so the
20+
# plan shows a spurious update of consumer -- but at deploy the direct engine downgrades it to a
21+
# no-op once ${referent.id} resolves to the unchanged value.
2122
#
2223
# This test runs the direct engine only. For context, the terraform engine (now being removed)
23-
# left consumer alone: it re-plans at apply, finds ${referent.id} resolved to the unchanged
24+
# also left consumer alone: it re-plans at apply, finds ${referent.id} resolved to the unchanged
2425
# value, and issues no request for consumer.
2526
resources:
2627
schemas:

‎acceptance/bundle/resources/schemas/recreate_id_ref_spurious_update/out.requests.direct.txt‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,6 @@
55
"force": "true"
66
}
77
}
8-
{
9-
"method": "PATCH",
10-
"path": "/api/2.1/unity-catalog/schemas/main.consumer",
11-
"body": {
12-
"comment": "refers to main.referent",
13-
"custom_max_retention_hours": 0
14-
}
15-
}
168
{
179
"method": "POST",
1810
"path": "/api/2.1/unity-catalog/schemas",

0 commit comments

Comments
 (0)