☁️ Azure Kusto Cluster Customer-Managed Key Terraform Module

Encrypts an Azure Data Explorer cluster's data at rest under a key you control (azurerm_kusto_cluster_customer_managed_key) — Key Vault or Managed HSM, with the exactly-one-of rule enforced at plan time. Targets hashicorp/azurerm ~> 4.0.

• 🔐 Declares one customer-managed key configuration for a Kusto cluster, as a keystone resource named this.
• 🧭 Decides who holds the key, not whether data is encrypted. Kusto encrypts at rest either way; this makes the key yours, which is a governance and revocation capability rather than an encryption upgrade.
• ✅ Simpler than the equivalent Service Bus module, and worth saying so. azurerm_kusto_cluster has no inline customer-managed-key block, so this separate resource is the only way to express the configuration — nothing competes with it and there is no perpetual diff to live with.
• 🚦 Enforces "exactly one of key_vault_id or managed_hsm_key_id" at plan time. Every field on this resource is optional in the schema, so neither being set is accepted at plan and fails at apply.
• 🔄 Leaves key_version null by default so a rotation inside Key Vault takes effect without a Terraform change.
• 🗝️ Takes references, never key material — nothing secret passes through configuration or state.

💡 Why it matters: The trade-off of customer-managed keys is rarely stated plainly, so here it is: you gain the ability to render the cluster's data unreadable at will, and you accept that a vault outage, a deleted key or an expired key does the same thing accidentally. The vault joins the cluster's availability path. That is worth doing deliberately — and it is why purge protection is a prerequisite rather than a suggestion.

If this module saves you time, please consider supporting its continued development:

• ⭐ Star the repository on GitHub.
• 🤝 Connect on LinkedIn: linkedin.com/in/microsoftexpert
• ☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

flowchart LR
  cluster["terraform-azurerm-kusto-cluster"]
  db["terraform-azurerm-kusto-database"]
  adc["terraform-azurerm-kusto-attached-database-configuration"]
  script["terraform-azurerm-kusto-script"]
  cosmos["terraform-azurerm-kusto-cosmosdb-data-connection"]
  eg["terraform-azurerm-kusto-eventgrid-data-connection"]
  eh["terraform-azurerm-kusto-eventhub-data-connection"]
  iot["terraform-azurerm-kusto-iothub-data-connection"]
  cmk["terraform-azurerm-kusto-cluster-customer-managed-key"]
  mpe["terraform-azurerm-kusto-cluster-managed-private-endpoint"]
  cpa["terraform-azurerm-kusto-cluster-principal-assignment"]
  dpa["terraform-azurerm-kusto-database-principal-assignment"]
  cluster -->|"hosts"| db
  cluster -->|"attaches leader db"| adc
  db -->|"target of"| script
  db -->|"ingests via"| cosmos
  db -->|"ingests via"| eg
  db -->|"ingests via"| eh
  db -->|"ingests via"| iot
  cluster -->|"encrypted by, BY ID"| cmk
  cluster -->|"private egress via, BY NAME"| mpe
  cluster -->|"data-plane roles: ALL databases"| cpa
  db -->|"data-plane roles: ONE database, prefer this"| dpa
  mpe -->|"makes the source reachable"| eh
  classDef me fill:#0078D4,stroke:#004578,color:#ffffff;
  classDef ext fill:#f2f2f2,stroke:#c8c8c8,color:#111111;
  class cmk me;
  class cluster,db,adc,script,cosmos,eg,eh,iot,mpe,cpa,dpa ext;

flowchart TB
  what["WHO HOLDS THE KEY, not whether data is encrypted: Kusto encrypts at rest either way"]
  simple["SIMPLER THAN THE SERVICE BUS EQUIVALENT: azurerm_kusto_cluster has NO inline customer_managed_key block, so nothing competes with this resource and there is NO perpetual diff"]
  xor["EXACTLY ONE of key_vault_id or managed_hsm_key_id. Every field is optional in the schema, so NEITHER is accepted at plan and fails at apply: rejected here at plan instead."]
  cycle["all three cross-field checks live on ONE variable, read one-directionally, to avoid a validation cycle"]
  kv["key_vault_id plus key_name: the Key Vault path"]
  hsm["managed_hsm_key_id: the FIPS 140-2 Level 3 single-tenant path, versionless or versioned"]
  ver["key_version left NULL is the better default: the cluster follows the current version so rotation needs no Terraform change"]
  ident["user_identity null means the cluster's SYSTEM-assigned identity reads the key"]
  grant["that identity still needs Get plus Wrap Key plus Unwrap Key: a SEPARATE grant to a DIFFERENT principal, and the most-missed step"]
  vault["purge protection and soft delete on the vault: REQUIRED, and the apply is refused without both"]
  avail["the vault joins the AVAILABILITY PATH of the cluster: revoke the key and the data is unreadable"]
  this["terraform-azurerm-kusto-cluster-customer-managed-key"]
  res["azurerm_kusto_cluster_customer_managed_key.this"]
  out["outputs: id which the provider sets to the CLUSTER id, plus derived posture booleans. The key reference is NOT re-emitted."]

  what -->|"read this first"| this
  simple -->|"good news"| this
  xor -->|"enforced at plan"| this
  cycle -->|"how"| xor
  kv -->|"path A"| xor
  hsm -->|"path B"| xor
  ver -->|"rotation"| kv
  ident -->|"who reads it"| this
  grant -->|"without it the apply fails"| ident
  vault -->|"prerequisite"| kv
  avail -->|"the cost of custody"| vault
  this -->|"creates"| res
  res -->|"emits"| out

  classDef me fill:#0078D4,stroke:#004578,color:#fff;
  classDef keystone fill:#004578,stroke:#001f3f,color:#fff;
  classDef sib fill:#eef2f7,stroke:#b8c4d0,color:#1b1b1b;
  class this me;
  class res keystone;
  class what,simple,xor,cycle,kv,hsm,ver,ident,grant,vault,avail,out sib;

Resource inventory

Schema notes that bite — confirmed against the live provider schema:

• cluster_id is force-new. Changing it creates a new encryption configuration rather than moving the existing one.
• Every other field is marked optional in the schema, including both key paths. So a call with neither key_vault_id nor managed_hsm_key_id passes the provider's own validation and fails at apply. This module rejects it at plan.
• Exactly one of key_vault_id / managed_hsm_key_id is legal. Both is a conflict; neither is the trap above.
• The id this resource exports is the cluster's Resource ID, not a distinct child ID — so an import block takes the cluster ID.
• azurerm_kusto_cluster has no inline customer_managed_key block. Unlike Service Bus, there is no second declaration of the same field and therefore no drift.
• user_identity null means the cluster's system-assigned identity is used.
• No tags and no name. The universal tail is timeouts only.
• Kusto cluster operations are slow; the provider's generous default timeouts are appropriate.

Two audiences, and conflating them causes most failed first applies.

The caller:

The identity that reads the key — the cluster's system-assigned identity by default, or user_identity:

• The Microsoft.Kusto and Microsoft.KeyVault resource providers registered on the subscription.
• An existing Kusto cluster with a managed identity. This module does not create or enable one.
• A Key Vault with purge protection and soft delete enabled, or a Managed HSM. 🔒 Both are required, not advisable: the provider reads the vault during create and refuses with "must be configured for both Purge Protection and Soft Delete" if either is off. That check lives in the provider's create path rather than in its schema, so it needs credentials and cannot fire offline — nothing local will warn you before the apply. The requirement is a good one: without soft delete, a deleted key would leave the cluster's data permanently unreadable.
• An existing RSA key in that vault or HSM.
• The reading identity granted key access, with time allowed for role-assignment propagation.

terraform-azurerm-kusto-cluster-customer-managed-key/
├── providers.tf   # required_version + the pinned azurerm provider. No provider block.
├── variables.tf   # cluster_id, the two key paths, key_name/key_version, user_identity, timeouts
├── main.tf        # the keystone azurerm_kusto_cluster_customer_managed_key.this
├── outputs.tf     # id (the cluster ID — see the note), cluster_id, three derived posture booleans
├── README.md      # this document
├── SCOPE.md       # the cross-module contract
├── LICENSE        # MIT
└── .gitignore

provider "azurerm" {
  features {}
}

module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id   = module.adx.id
  key_vault_id = module.kv.id
  key_name     = azurerm_key_vault_key.adx.name
}

ℹ️ The caller configures the provider, its authentication, and the mandatory features {} block. This module declares none of them.

⚠️ Before running this: the cluster's identity needs Get, Wrap Key and Unwrap Key on the key. Example 4 shows the grant — it is the step most first attempts miss.

Consumes

Emits

The examples below reference existing resources by ID or name rather than creating them; this module owns only its own resource. Those references are declared inputs:

variable "adx_prod_id" {
  description = "id of an existing adx prod that these examples reference but do not create."
  type        = string
}

variable "adx_staging_id" {
  description = "id of an existing adx staging that these examples reference but do not create."
  type        = string
}

The examples read the current tenant from the provider rather than hard-coding it.

data "azurerm_client_config" "current" {}

module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id   = module.adx.id
  key_vault_id = module.kv.id
  key_name     = azurerm_key_vault_key.adx.name
}

module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id         = module.adx.id
  managed_hsm_key_id = var.adx_hsm_key_id
}

# ✅ Preferred: follows the key's current version.
module "adx_cmk" {
  # ...
  key_name = azurerm_key_vault_key.adx.name
  # key_version deliberately omitted
}

# ⚠️ Pinned: every rotation is now a configuration change someone must remember.
module "adx_cmk" {
  # ...
  key_name    = azurerm_key_vault_key.adx.name
  key_version = azurerm_key_vault_key.adx.version
}

# The CLUSTER's identity must be able to use the key. This is a DIFFERENT principal
# from the one running Terraform, and it is a separate grant.
resource "azurerm_role_assignment" "adx_wrap_unwrap" {
  scope                = azurerm_key_vault_key.adx.resource_versionless_id
  role_definition_name = "Key Vault Crypto Service Encryption User"
  principal_id         = module.adx.identity_principal_id
}

module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id   = module.adx.id
  key_vault_id = module.kv.id
  key_name     = azurerm_key_vault_key.adx.name

  depends_on = [azurerm_role_assignment.adx_wrap_unwrap]
}

module "kv" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-key-vault.git?ref=v1.0.0"

  tenant_id = data.azurerm_client_config.current.tenant_id

  name                = "kv-adx-prod"
  resource_group_name = module.rg.name
  location            = module.rg.location

  purge_protection_enabled   = true # non-negotiable for a vault backing encryption
  soft_delete_retention_days = 90
}

module "adx_identity" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-user-assigned-identity.git?ref=v1.0.0"

  name                = "id-adx-cmk"
  resource_group_name = module.rg.name
  location            = module.rg.location
}

module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id    = module.adx.id
  key_vault_id  = module.kv.id
  key_name      = azurerm_key_vault_key.adx.name
  user_identity = module.adx_identity.id
}

# ❌ Neither path set. The provider's schema ACCEPTS this and fails at apply.
module "adx_cmk" {
  cluster_id = module.adx.id
}

Error: Invalid value for variable

  Exactly one of key_vault_id or managed_hsm_key_id must be set. Supplying both is
  a conflict; supplying neither is accepted by the provider's schema and then fails
  at apply, so it is rejected here instead.

# ❌ Key Vault path with no key name.
module "adx_cmk" {
  cluster_id   = module.adx.id
  key_vault_id = module.kv.id
}

Error: Invalid value for variable

  key_name is required when key_vault_id is set — a vault reference alone does not
  identify a key.

output "adx_cmk_id" {
  value = module.adx_cmk.id
}

adx_cmk_id = "/subscriptions/00000000-.../resourceGroups/rg-analytics/providers/Microsoft.Kusto/clusters/adxprod"

import {
  to = module.adx_cmk.azurerm_kusto_cluster_customer_managed_key.this
  id = "/subscriptions/00000000-.../providers/Microsoft.Kusto/clusters/adxprod"
}

# Second and subsequent plans, from an unchanged configuration:
No changes. Your infrastructure matches the configuration.

# Disabling the key in Key Vault renders the cluster's data unreadable.
# This is deliberate — it is the capability a customer-managed key buys you.

locals {
  clusters = {
    prod    = var.adx_prod_id
    staging = var.adx_staging_id
  }
}

module "adx_cmk" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"
  for_each = local.clusters

  cluster_id   = each.value
  key_vault_id = module.kv.id
  key_name     = azurerm_key_vault_key.per_cluster[each.key].name
}

output "adx_encryption_posture" {
  value = {
    cluster            = module.adx_cmk.cluster_id
    in_managed_hsm     = module.adx_cmk.uses_managed_hsm
    version_pinned     = module.adx_cmk.key_version_pinned     # expect false
    system_identity    = module.adx_cmk.uses_system_assigned_identity
  }
}

provider "azurerm" {
  features {}
}

module "rg" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"

  name     = "rg-analytics-prod"
  location = "eastus"
}

# 1 · A vault that can safely back encryption at rest.
module "kv" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-key-vault.git?ref=v1.0.0"

  tenant_id = data.azurerm_client_config.current.tenant_id

  name                = "kv-adx-prod"
  resource_group_name = module.rg.name
  location            = module.rg.location

  purge_protection_enabled   = true
  soft_delete_retention_days = 90
}

resource "azurerm_key_vault_key" "adx" {
  name         = "adx-cmk"
  key_vault_id = module.kv.id
  key_type     = "RSA"
  key_size     = 2048
  key_opts     = ["unwrapKey", "wrapKey"] # only what encryption at rest needs
}

# 2 · The cluster, with its default system-assigned identity.
module "adx" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster.git?ref=v1.0.0"

  name                = "adxprod"
  resource_group_name = module.rg.name
  location            = module.rg.location

  sku = {
    name     = "Standard_E8ads_v5"
    capacity = 2
  }
}

# 3 · The cluster's identity must be able to wrap and unwrap with the key.
resource "azurerm_role_assignment" "adx_wrap_unwrap" {
  scope                = azurerm_key_vault_key.adx.resource_versionless_id
  role_definition_name = "Key Vault Crypto Service Encryption User"
  principal_id         = module.adx.identity_principal_id
}

# 4 · Link the key. This is the module.
module "adx_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster-customer-managed-key.git?ref=v1.0.0"

  cluster_id   = module.adx.id
  key_vault_id = module.kv.id
  key_name     = azurerm_key_vault_key.adx.name

  depends_on = [azurerm_role_assignment.adx_wrap_unwrap]
}

output "analytics_encryption" {
  value = {
    cluster        = module.adx_cmk.cluster_id
    version_pinned = module.adx_cmk.key_version_pinned
  }
}

variable "cluster_id" {
  type = string
  # ⚠️ Force-new.
}

variable "key_vault_id" {
  type    = string
  default = null
  # Carries all three cross-field checks (one-directionally, to avoid a validation cycle):
  #   1. exactly one of key_vault_id / managed_hsm_key_id
  #   2. key_name required when key_vault_id is set
  #   3. key_name rejected on the managed-HSM path (key_version is ACCEPTED there but ignored,
  #      and reported through an output rather than refused)
  # 🔒 A reference to a vault, not key material. Purge protection + soft delete FIRST.
}

variable "key_name" {
  type    = string
  default = null
}

variable "key_version" {
  type    = string
  default = null
  # 💡 Null follows the key's current version, so rotation needs no Terraform change.
}

variable "managed_hsm_key_id" {
  type    = string
  default = null
  # The FIPS 140-2 Level 3 single-tenant path. Versioned, so rotation IS a config change.
}

variable "user_identity" {
  type    = string
  default = null
  # Null = the cluster's system-assigned identity. Either way the identity needs
  # Get + Wrap Key + Unwrap Key on the key — a separate grant to a different principal.
}

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

🔒 No secrets here, and none possible: this module handles key references, never key material. The key reference is deliberately not re-emitted — Key Vault is the source of truth for the key, not this module.

• The schema's optionality is the real hazard. Every field except cluster_id is optional, so the provider will accept a call that specifies no key at all and fail only at apply. That is precisely the class of error this library's typing rules exist to catch, so the exactly-one-of rule is enforced at plan.

• All three cross-field checks live on one variable. Terraform rejects validations that reference each other across variables — the resulting error is a cycle, not a helpful message. Putting all three on key_vault_id and reading the others one-directionally avoids it. This is the same pattern the authorization-rule modules in this suite use, and the reason is worth remembering rather than rediscovering.

• No inline block means no drift, and that is not universal. The equivalent Service Bus resource has an inline customer_managed_key block on its namespace, so a separately-declared key produces a permanent diff. azurerm_kusto_cluster has no such block. If you are carrying a mental model over from that module, this is the difference.

• Unpinned key_version is a deliberate default, not laziness. Following the key's current version means Key Vault owns rotation. Pinning moves rotation into Terraform, where it becomes a change someone has to remember — and until they do, the cluster silently keeps using superseded key material.

• The identity grant is the most common first-apply failure, and it is easy to misdiagnose because the error reads like a wrong key ID. It is a grant to the cluster's identity, not the caller's, and role assignments propagate asynchronously.

• The Key Vault becomes a production dependency of the cluster. A firewall change, a deleted key or an expired key can each make data unreadable. Encryption custody and analytics availability are now coupled.

• The id quirk is worth internalizing. The provider exports this resource's id as the cluster's Resource ID. State reads and imports both behave accordingly.

• Before applying: confirm the cluster has an identity, and that the identity has key access.
• Before applying: confirm purge protection and soft delete are on the vault.
• Before applying: decide whether key_version should be pinned, and default to not pinning it.

terraform init -backend=false
terraform validate
terraform fmt -check

• Pin the source to a tag — ?ref=v1.0.0 — never a branch.
• Plan-only from here. A human applies from CI.
• ⚠️ Order on first apply: cluster and identity, then the key grant, then this module. Example 4 shows the depends_on.
• ℹ️ Subsequent plans are clean — there is no perpetual diff on this path.
• To import an existing configuration, use the cluster's Resource ID.

terraform validate and terraform fmt -check are the offline gate. They confirm:

• cluster_id is present and typed string;
• the timeouts value is an object of Go duration strings - but note that a key the type does not declare is SILENTLY DISCARDED rather than refused, so a misspelling here produces no error anywhere and the provider default quietly applies;
• no validation cycle exists among the cross-field checks (a cycle is a validate-time error);
• the module declares no provider block.

Every validation {} here was verified by driving deliberately bad values through the conditions directly, because terraform validate does not evaluate module-input validations — a broken condition would otherwise ship clean and fail in a caller's plan. All 21 cases behave as intended, and roughly a third of them are positive controls: a check that is too strict is as much a defect as one that is missing, since a failed validation blocks terraform destroy as well as apply. Accepted: both key paths in their valid shapes, a versioned and a versionless HSM key URI, a sovereign-cloud HSM host, a valid user-assigned identity, and a compound duration such as 1h15m. Rejected: both paths at once, neither path, a vault without key_name, key_name on the HSM path, a database ID or a Key Vault ID passed as cluster_id, a key URI passed where a Resource ID or a bare name belongs, blank strings, a principal GUID passed as user_identity, and a bare number passed as a timeout.

The two outputs that are not constants — key_rotation_is_automatic and key_version_pinned — were separately evaluated across all six combinations of key path and version pinning, because terraform validate never evaluates an output expression at all.

What only plan and apply exercise:

• whether the cluster has a managed identity at all;
• whether that identity holds Get / Wrap Key / Unwrap Key on the key, and whether the grant has propagated;
• whether the key type and size are supported;
• whether the vault's firewall permits the cluster to reach it.

Outputs:

cluster_id                    = "/subscriptions/00000000-.../providers/Microsoft.Kusto/clusters/adxprod"
id                            = "/subscriptions/00000000-.../providers/Microsoft.Kusto/clusters/adxprod"
key_version_pinned            = false
uses_managed_hsm              = false
uses_system_assigned_identity = true

ℹ️ id and cluster_id being identical is the provider quirk, not a copy-paste error in this document.

• azurerm_kusto_cluster_customer_managed_key — provider documentation for this resource.
• azurerm_kusto_cluster — the cluster and its managed identity. Note it has no inline customer-managed-key block.
• azurerm_key_vault_key — the key this module references.
• Customer-managed keys in Azure Data Explorer — the service's own account of key custody and revocation.
• Sibling modules in this family: terraform-azurerm-kusto-cluster, terraform-azurerm-kusto-database, terraform-azurerm-kusto-cluster-managed-private-endpoint, terraform-azurerm-kusto-cluster-principal-assignment, terraform-azurerm-kusto-database-principal-assignment, plus terraform-azurerm-key-vault.
• This module's SCOPE.md.

💙 "Infrastructure as Code should be standardized, consistent, and secure."