☁️ Azure Data Factory Linked Service — Key Vault Terraform Module

The pointer that lets every other linked service in a Data Factory avoid holding a secret, targeting hashicorp/azurerm ~> 4.0.

• 🗝️ Creates one azurerm_data_factory_linked_service_key_vault — the resource every key_vault_password, key_vault_connection_string, key_vault_license and key_vault_secret_reference block in the factory references by name.
• 🔒 It holds no secret at all, and that is the point. Nothing here takes a credential and nothing here returns one. Its whole job is to let siblings name a secret instead of carrying one.
• 🔴 A vault in a different subscription from the factory never converges. Data Factory stores the vault's base URL, not its Resource ID, and the read resolves it back by listing vaults in the factory's subscription — finding nothing, reporting no error, and writing key_vault_id back empty.
• 🔴 key_vault_id is required and not force-new. Repointing it is an in-place update that silently redirects every secret reference in every sibling.
• ⚠️ Creating this grants nothing. The factory's identity still needs read access on the vault, and nothing in this module creates it.

💡 Why it matters: this is the one linked service whose refresh behaviour depends on where the vault lives and what the caller may list — not on anything visible in its own configuration.

If this module saved you time:

• ⭐ Star the repository — it is the cheapest signal that this work is worth continuing.
• 💼 Connect on LinkedIn: linkedin.com/in/microsoftexpert
• ☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

flowchart TB
  RG["terraform-azurerm-resource-group"]
  ADF["terraform-azurerm-data-factory"]
  KV["terraform-azurerm-key-vault"]
  THIS["terraform-azurerm-data-factory-linked-service-key-vault"]
  GRANT["terraform-azurerm-role-assignments, the factory identity on the vault, NOT created here"]
  SQL["terraform-azurerm-data-factory-linked-service-azure-sql-database"]
  SRV["terraform-azurerm-data-factory-linked-service-sql-server"]
  SSIS["terraform-azurerm-data-factory-integration-runtime-azure-ssis"]

  RG -->|"name"| ADF
  ADF -->|"id"| THIS
  KV -->|"id, and its base URL is what Data Factory actually stores"| THIS
  ADF -->|"identity_principal_id"| GRANT
  GRANT -->|"read access on the vault, without which every reference below fails"| KV
  THIS -->|"name, into key_vault_connection_string and key_vault_password"| SQL
  THIS -->|"name, into key_vault_connection_string and key_vault_password"| SRV
  THIS -->|"name, into key_vault_password and key_vault_license"| SSIS

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef keystone fill:#004578,stroke:#00243d,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class ADF keystone
  class RG,KV,GRANT,SQL,SRV,SSIS sibling

The three edges leaving this module are the reason it exists — and the edge that does not leave it is the grant. Creating the linked service tells the factory where the vault is; a separate role assignment is what lets it read anything.

flowchart TB
  subgraph INPUTS["Inputs"]
    ID["name and data_factory_id, THE ONLY TWO FORCE-NEW FIELDS"]
    VAULT["key_vault_id, REQUIRED and NOT force-new"]
    META["integration_runtime_name, description, parameters, annotations, additional_properties"]
  end

  THIS["azurerm_data_factory_linked_service_key_vault.this"]

  subgraph OUTPUTS["Outputs"]
    DIFF["a_key_vault_in_another_subscription_produces_a_permanent_diff"]
    URL["data_factory stores the vault URL, not its resource id"]
    PERM["reading this requires permission to LIST key vaults in the factory subscription"]
    REPOINT["repointing_the_vault_is_an_in_place_update, redirecting every sibling silently"]
    GRANT["this linked service grants NO access to the vault"]
    NAMEOUT["name, consumed by every secret bearing sibling BY NAME"]
  end

  ID --> THIS
  VAULT --> THIS
  META --> THIS

  VAULT --> DIFF
  ID --> DIFF
  THIS --> URL
  THIS --> PERM
  VAULT --> REPOINT
  THIS --> GRANT
  THIS --> NAMEOUT

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class ID,VAULT,META,DIFF,URL,PERM,REPOINT,GRANT,NAMEOUT sibling

Schema notes that bite — verified against the live provider source:

• 🔴 Data Factory stores the vault's base URL, not its Resource ID. The provider derives the URL from the ID on create, and on every read reconstructs the ID from the URL by listing the Key Vaults in the factory's subscription.
• 🔴 A vault outside that subscription is never found, and the lookup returns no error. The helper returns nothing with the comment that callers must handle it separately — and this resource does not, so key_vault_id is written back empty and every plan afterwards proposes to set it again.
• 🔴 Refreshing therefore requires permission to LIST Key Vaults across the factory's subscription, which is broader than reading the Data Factory.
• 🔴 key_vault_id is Required and NOT force-new. It is declared through a shared helper that is Required without ForceNew — a ForceNew variant of that helper exists and this resource does not use it.
• ⚠️ The name validator is nearly inert. Its expression is anchored end to end, so it refuses only a value composed entirely of the punctuation its message lists. a/b, 100%done, a Resource ID and the empty string all pass. The same validator is shared by the whole linked-service and dataset family.
• ⚠️ additional_properties is validated by nothing; a misspelled key is silently ignored.
• ⚠️ annotations is an ordered list, so reordering is a real diff — and it is not tags.
• ⚠️ Only name and data_factory_id are force-new. There is no location and no tags.
• ℹ️ The Resource ID's segment is lower-case linkedservices.
• ℹ️ No cross-field rules and no CustomizeDiff, so every provider check here fires at terraform validate, offline.
• ℹ️ The requires-import error names this resource correctly.

🔴 The Terraform principal needs more than Data Factory Contributor here. Refreshing this resource enumerates the Key Vaults in the factory's subscription, so a principal that can manage the factory but not list vaults will not populate key_vault_id — and the resource will appear to drift on every plan.

⚠️ Separately, the FACTORY'S identity needs read access on the vault, through a Key Vault role assignment or an access policy. That is a different principal, a different scope, and nothing in this module creates it.

• An existing Data Factory.
• An existing Key Vault — preferably in the same subscription as the factory; see the caveat above.
• A read grant for the factory's identity on that vault, made elsewhere.
• For a vault restricted to a private endpoint: a self-hosted or managed-virtual-network integration runtime, named in integration_runtime_name.

terraform-azurerm-data-factory-linked-service-key-vault/
├── providers.tf    # required_version + the pinned azurerm; no provider block
├── variables.tf    # 9 inputs, 13 validations, none sensitive -- there is no secret here
├── main.tf         # the keystone, plus the cross-subscription derivation
├── outputs.tf      # 40 outputs; none is sensitive, because none can be
├── README.md       # this file
├── SCOPE.md        # the cross-module contract
├── LICENSE         # MIT
└── .gitignore

module "keyvault_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-key-vault.git?ref=v1.0.0"

  name            = "ls-keyvault"
  data_factory_id = module.data_factory.id
  key_vault_id    = module.key_vault.id
}

The caller configures the provider, its authentication, and the mandatory features {} block:

provider "azurerm" {
  features {}
}

💡 Then pass module.keyvault_link.name into every sibling's key_vault_* block. That name — not the Resource ID — is what those blocks take.

Consumes

Emits

module "kv_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-key-vault.git?ref=v1.0.0"

  name            = "ls-keyvault"
  data_factory_id = module.data_factory.id
  key_vault_id    = module.key_vault.id
}

module "sql_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-azure-sql-database.git?ref=v1.0.0"

  name            = "ls-sales-db"
  data_factory_id = module.data_factory.id

  key_vault_connection_string = {
    linked_service_name = module.kv_link.name # a NAME, never a Resource ID
    secret_name         = "sales-db-connection"
  }
}

# The factory is in subscription A; the vault is in subscription B.
module "kv_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-key-vault.git?ref=v1.0.0"

  name            = "ls-shared-keyvault"
  data_factory_id = module.data_factory.id # /subscriptions/AAAA-.../factories/adf-corp
  key_vault_id    = "/subscriptions/BBBB-0000-0000-0000-000000000000/resourceGroups/rg-security/providers/Microsoft.KeyVault/vaults/kv-shared"
}

data_factory_id = module.data_factory.id # rg-data-eastus
key_vault_id    = "/subscriptions/AAAA-0000-0000-0000-000000000000/resourceGroups/rg-security/providers/Microsoft.KeyVault/vaults/kv-shared" # rg-security, SAME subscription

# Changing this is NOT a replacement. It is an in-place update.
key_vault_id = "/subscriptions/AAAA-0000-0000-0000-000000000000/resourceGroups/rg-security/providers/Microsoft.KeyVault/vaults/kv-replacement"

module "kv_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-key-vault.git?ref=v1.0.0"

  name            = "ls-keyvault"
  data_factory_id = module.data_factory.id
  key_vault_id    = module.key_vault.id
}

# WITHOUT THIS, every secret reference through the linked service above fails at run time.
module "vault_grant" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"

  scope = module.key_vault.id

  role_assignments = {
    factory_reads_secrets = {
      principal_id         = module.data_factory.identity_principal_id
      role_definition_name = "Key Vault Secrets User"
      principal_type       = "ServicePrincipal"
    }
  }
}

integration_runtime_name = "ir-vnet" # a NAME, from a managed-vnet or self-hosted runtime module

# Each of these is rejected at `terraform plan`, offline, with a targeted message:
#   key_vault_id = ".../vaults/kv-corp/secrets/sql-password"  -> a SECRET id is too long
#   key_vault_id = "https://kv-corp.vault.azure.net/"         -> that is the BASE URL, not the id
#   key_vault_id = "kv-corp"                                   -> a bare name
#   key_vault_id = module.data_factory.id                      -> the wrong resource type

key_vault_id = module.key_vault.id # correct

name = "ls-keyvault"   # accepted, despite '-' being in the "not allowed" list
name = "ls.kv/shared"  # also accepted, despite '.' and '/'
# name = "---"         # REFUSED -- the one shape the provider's validator actually catches
# name = ""            # refused BY THIS MODULE; the provider accepts it

locals {
  databases = {
    sales   = "sales-db-connection"
    finance = "finance-db-connection"
    hr      = "hr-db-connection"
  }
}

module "db_link" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-azure-sql-database.git?ref=v1.0.0"
  for_each = local.databases

  name            = "ls-${each.key}-db"
  data_factory_id = module.data_factory.id

  key_vault_connection_string = {
    linked_service_name = module.kv_link.name
    secret_name         = each.value
  }
}

annotations = ["owner:data-platform", "scope:secrets"]

import {
  to = module.kv_link.azurerm_data_factory_linked_service_key_vault.this
  id = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-data-eastus/providers/Microsoft.DataFactory/factories/adf-corp/linkedservices/ls-keyvault"
}

check "the_vault_is_in_the_factory_subscription" {
  assert {
    condition     = !module.kv_link.a_key_vault_in_another_subscription_produces_a_permanent_diff
    error_message = "The vault is in another subscription; the provider cannot resolve it back and this resource will never converge."
  }
}

check "a_runtime_that_can_reach_a_private_vault" {
  assert {
    condition     = !module.kv_link.runs_on_the_factory_default_integration_runtime
    error_message = "No integration runtime was named, so this resolves to the public-network default."
  }
}

module "resource_group" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"

  name     = "rg-data-eastus"
  location = "eastus"
}

module "key_vault" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-key-vault.git?ref=v1.0.0"

  name                = "kv-data-eastus"
  resource_group_name = module.resource_group.name
  location            = module.resource_group.location
  tenant_id           = var.tenant_id
}

module "data_factory" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory.git?ref=v1.0.0"

  name                   = "adf-corp-eastus"
  resource_group_name    = module.resource_group.name
  location               = module.resource_group.location
  public_network_enabled = false
}

# The pointer.
module "kv_link" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-key-vault.git?ref=v1.0.0"

  name            = "ls-keyvault"
  data_factory_id = module.data_factory.id
  key_vault_id    = module.key_vault.id
  description     = "Secret source for every credential-bearing linked service in this factory."
}

# The grant, without which the pointer resolves to nothing usable.
module "vault_grant" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"

  scope = module.key_vault.id

  role_assignments = {
    factory_reads_secrets = {
      principal_id         = module.data_factory.identity_principal_id
      role_definition_name = "Key Vault Secrets User"
      principal_type       = "ServicePrincipal"
    }
  }
}

# A consumer, carrying no secret of its own.
module "sales_db" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-linked-service-azure-sql-database.git?ref=v1.0.0"

  name            = "ls-sales-db"
  data_factory_id = module.data_factory.id

  key_vault_connection_string = {
    linked_service_name = module.kv_link.name
    secret_name         = "sales-db-connection"
  }
}

check "the_vault_is_in_the_factory_subscription" {
  assert {
    condition     = !module.kv_link.a_key_vault_in_another_subscription_produces_a_permanent_diff
    error_message = "The vault is in another subscription and this resource will never converge."
  }
}

output "key_vault_linked_service_name" {
  value = module.kv_link.name
}

Identity (both force-new): name, data_factory_id. The vault: key_vault_id — required, and not force-new. Metadata: description, integration_runtime_name, parameters, annotations, additional_properties, timeouts.

There is no tags input — the resource exposes none, and annotations are not tags. No input is sensitive, because this resource accepts no secret.

variable "name" { type = string }            # required, force-new; consumed BY NAME by every sibling
variable "data_factory_id" { type = string } # required, force-new
variable "key_vault_id" { type = string }    # required, NOT force-new

variable "description" { type = string, default = null }
variable "integration_runtime_name" { type = string, default = null } # a NAME
variable "parameters" { type = map(string), default = {} }
variable "annotations" { type = list(string), default = [] }          # ORDERED; not tags
variable "additional_properties" { type = map(string), default = {} } # validated by nothing

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

This resource is a pointer, and it is the reason the rest of the family can be secret-free. A key_vault_password on a SQL linked service, a key_vault_license on the SSIS runtime, a key_vault_secret_reference anywhere — all of them name this linked service and a secret name, and the value is fetched by the service at run time. Nothing here takes a credential and nothing here returns one. That is why no input and no output in this module is marked sensitive: there is nothing to mark.

The identifier round-trip does not round-trip, and that is where the trouble is. Data Factory persists the vault's base URL — https://<vault>.vault.azure.net/ — and not its Resource ID. On create, the provider derives the URL from the ID you supply. On every read it must go the other way, and it does so by listing the Key Vaults in the factory's subscription, caching them by name, and matching the URL against that cache. Two consequences follow, and neither is visible in the resource's own configuration. First, refreshing requires permission to list vaults across that subscription — broader than Data Factory Contributor. Second, a vault in a different subscription is never found: the lookup returns nothing and no error, so the provider writes key_vault_id back as empty, and every subsequent plan proposes to set a required argument that the configuration already sets correctly. The linked service works perfectly in Azure the entire time. a_key_vault_in_another_subscription_produces_a_permanent_diff is derived from the two Resource IDs, so it is true at plan time, before anything is created.

A required reference that is not force-new is unusual, and here it has reach. key_vault_id is declared through a shared helper that is Required without ForceNew; a ...RequiredForceNew variant of that same helper exists, and this resource does not use it. So repointing the linked service at a different vault is an in-place update — and it silently redirects every secret reference in every sibling that names this linked service. Those siblings' plans show no change, because nothing about them changed. Nothing in Terraform connects the two facts.

Creating this grants nothing. The linked service tells the factory where the vault is; the factory's identity still needs a Key Vault role assignment, or an access policy on a vault still using access policies. That is a different principal at a different scope and this module does not create it. A configuration that looks complete therefore fails at first use rather than at apply — which is the single most common way this resource disappoints.

The name is load-bearing in a way it is not elsewhere. It is force-new, it is what every sibling references, and the provider's own validator for it is nearly inert — refusing a value only when it consists of nothing but the punctuation its message forbids. The module mirrors that expression rather than the message, adds the non-empty and not-a-Resource-ID checks the provider omits, and reports the rest.

Secure-by-default applies cleanly here for once: the empty call carries no secret because the resource cannot carry one. The rules that do bite are the two the module reports rather than refuses — the cross-subscription placement and the in-place vault change — because the provider accepts both, and a failing validation {} would block terraform destroy for anyone who already owns such a linked service.

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module at a tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI.

🔴 Before the first apply, check that the vault and the factory share a subscription, and that the Terraform principal can list Key Vaults in it. Both are refresh-time concerns that no amount of configuration review will surface later.

terraform validate covers every check the provider applies here — this resource declares no cross-field rules and no CustomizeDiff at all.

Offline, without credentials, the module's proof harness confirms:

• every wrong shape of vault reference is refused with a targeted message — a secret ID, a factory ID, an unrelated resource, a bare name, and the base URL, which earns its own message because Data Factory really does store the base URL internally;
• the cross-subscription condition has one fixture per branch, including the same-subscription-different- resource-group case that must not be flagged, and a casing-only variant that must not be either;
• 🔴 the force-new claim is read from the provider's Go source, not asserted — the harness requires that the resource uses commonschema.ResourceIDReferenceRequired, that the ...RequiredForceNew variant does not appear, and that exactly two ForceNew fields exist;
• 🔴 the base-URL round-trip and its subscription scoping are likewise read from the source, including that the lookup is scoped by the factory's own subscription;
• every name the provider's own message disowns is accepted, and only the all-punctuation shapes refused;
• the schema really does mark nothing on this resource sensitive, and the module marks nothing either;
• force_new_fields omits key_vault_id, which is the whole point of the finding.

Only terraform plan and apply exercise the ARM call. Nothing at any stage verifies that the vault exists, that a named secret is in it, or that the factory can read it.

id                                    = "/subscriptions/00000000-.../factories/adf-corp/linkedservices/ls-keyvault"
name                                  = "ls-keyvault"
data_factory_name                     = "adf-corp"
subscription_id                       = "8f3a2b1c-4d5e-6f70-8192-a3b4c5d6e7f8"
key_vault_name                        = "kv-data-eastus"
key_vault_subscription_id             = "8f3a2b1c-4d5e-6f70-8192-a3b4c5d6e7f8"
key_vault_is_in_another_resource_group = false
a_key_vault_in_another_subscription_produces_a_permanent_diff = false
runs_on_the_factory_default_integration_runtime = true
force_new_fields                      = ["name", "data_factory_id"]
this_linked_service_grants_no_access_to_the_vault = true
no_secret_is_accepted_or_emitted_by_this_module   = true

• azurerm_data_factory_linked_service_key_vault
• azurerm_data_factory_linked_service_azure_sql_database — a typical consumer
• Store credentials in Azure Key Vault — the pattern this resource enables
• Managed identity for Data Factory — the identity that needs the grant
• Azure Key Vault RBAC guide — Key Vault Secrets User and friends
• Data Factory naming rules — the real authority on names
• SCOPE.md — this module's cross-module contract

💙 "Infrastructure as Code should be standardized, consistent, and secure."