The data-history connection (
azurerm_digital_twins_time_series_database_connection): twin property updates flow through an Event Hub into an Azure Data Explorer table. The one member of this family that holds no secrets at all. Targetshashicorp/azurerm ~> 4.0.
- 🕰️ Declares the data-history connection for a Digital Twins instance, as a keystone resource named
this. A twin holds only its current property values — without this, there is no record of what a property was an hour ago. - 🔐 The identity-based member of this family, and the contrast is the point. It accepts no keys and no connection strings: nothing secret passes through configuration, plan output, or state. Authentication is the instance's own system-assigned managed identity.
- 🎫 So the security work moves from handling credentials to granting roles — including two grants that should be reduced once setup is done, which is the step most often skipped.
- 🧊 Every field is force-new, including the two optional ones. There is no update operation, so the module's
timeoutsobject carries noupdate. - 🚧 Documents the ordering trap up front: the Kusto cluster needs public network access enabled while the connection is created, because Digital Twins reaches in to build the table.
- 📐 Validates the two URI inputs, which are both easy to get backwards and fail only at apply.
💡 Why it matters: This module is cheap to write and easy to get wrong in ways Terraform cannot see. Four of its eight inputs describe two resources twice — once as a Resource ID, once as a URI — and nothing checks that the pairs agree. One of those URIs must use
sb://rather thanhttps://. And the connection will not create at all if the target cluster is locked down, which is the opposite of the order most teams would naturally do things in.
If this module saves you time, please consider supporting its continued development:
- ⭐ Star the repository on GitHub.
- 🤝 Connect on LinkedIn: linkedin.com/in/microsoftexpert
- ☕ Buy me a coffee: buymeacoffee.com/microsoftexpert
flowchart LR
rg["terraform-azurerm-resource-group"]
uai["terraform-azurerm-user-assigned-identity: optional, via identity_ids"]
dt["terraform-azurerm-digital-twins-instance: the family keystone, owns the system-assigned identity"]
eg["terraform-azurerm-digital-twins-endpoint-eventgrid: KEY-BASED, no identity option EXISTS"]
eh["terraform-azurerm-digital-twins-endpoint-eventhub: KEY-BASED, provider gap only"]
sb["terraform-azurerm-digital-twins-endpoint-servicebus: KEY-BASED, provider gap only, TOPIC not queue"]
ts["terraform-azurerm-digital-twins-time-series-database-connection: IDENTITY-BASED, no secrets at all"]
egt["terraform-azurerm-eventgrid-topic: keys read via a data source, the module does not emit them"]
ehar["terraform-azurerm-eventhub-authorization-rule: SEND only"]
sbar["terraform-azurerm-servicebus-topic-authorization-rule: SEND only"]
ehns["terraform-azurerm-eventhub-namespace: the hop for data history"]
kc["terraform-azurerm-kusto-cluster"]
kd["terraform-azurerm-kusto-database: must already exist, the table is created FOR you"]
sa["terraform-azurerm-storage-account: dead-letter container plus a SAS token"]
ra["terraform-azurerm-role-assignments: grants the DT identity its roles, reducible after setup"]
routes["event routes in the twin graph: DATA PLANE, name the endpoint, not modelled by Terraform"]
rg -->|"resource_group_name, location"| dt
uai -->|"identity_ids"| dt
dt -->|"id, BY ID"| eg
dt -->|"id, BY ID"| eh
dt -->|"id, BY ID"| sb
dt -->|"id, BY ID"| ts
egt -->|"endpoint URL plus TWO access keys"| eg
ehar -->|"two connection strings"| eh
sbar -->|"two connection strings"| sb
ehns -->|"id, name, and an sb:// URI you COMPOSE"| ts
kc -->|"id plus uri"| ts
kd -->|"name"| ts
sa -->|"dead_letter_storage_secret"| eg
sa -->|"dead_letter_storage_secret"| eh
sa -->|"dead_letter_storage_secret"| sb
dt -->|"identity_principal_id"| ra
ra -->|"Event Hubs Data Owner, Kusto Contributor, database Admin"| ts
eg -->|"name"| routes
eh -->|"name"| routes
sb -->|"name"| routes
classDef me fill:#0078D4,stroke:#004578,color:#fff;
classDef keystone fill:#004578,stroke:#001f3f,color:#fff;
classDef sib fill:#eef2f7,stroke:#b8c4d0,color:#1b1b1b;
class dt keystone;
class eg,eh,sb,ts me;
class rg,uai,egt,ehar,sbar,ehns,kc,kd,sa,ra,routes sib;
flowchart TB
what["WHAT IT BUYS: a twin holds only its CURRENT property values. This records the history."]
ident["THE IDENTITY-BASED MEMBER OF THIS FAMILY: no keys, no connection strings, nothing secret in config, plan output, or state"]
needs["so the instance needs a SYSTEM-ASSIGNED managed identity, enabled on the instance module, before this exists"]
setup["setup grants to that identity: Event Hubs Data Owner, Kusto Contributor, and a database principal assignment with role Admin"]
reduce["AFTER SETUP reduce to just Event Hubs Data Sender: the least-privilege end state, and easy to forget"]
pna["ORDERING TRAP: the Kusto cluster must have PUBLIC NETWORK ACCESS ENABLED during creation, because DT reaches in to create the table. Restrict the network AFTERWARDS."]
fail["with it disabled, setup fails with an internal-server-error that does not name the cause"]
fn["EVERY field is FORCE-NEW, including the two optional ones, so there is no update timeout at all"]
dbl["the namespace and the cluster are each supplied TWICE: a Resource ID and a URI. Nothing checks they agree."]
sburi["eventhub_namespace_endpoint_uri must use sb://, and no module output exists so you COMPOSE it from the name"]
kuri["kusto_cluster_uri is the cluster's uri output, not data_ingestion_uri and not the Resource ID"]
db["the DATABASE must already exist. DT creates the TABLE inside it, plus its ingestion mapping."]
tbl["changing kusto_table_name replaces the connection and LEAVES THE OLD TABLE behind holding the history so far"]
subset["the provider exposes a SUBSET of the ARM API: no twin-lifecycle or relationship-lifecycle tables, no removal recording, no user-assigned identity"]
this["terraform-azurerm-digital-twins-time-series-database-connection"]
res["azurerm_digital_twins_time_series_database_connection.this"]
out["outputs: id, name, plus the destination in the three pieces a Kusto query needs. No sensitive outputs, because there are no secrets."]
what -->|"read this first"| this
ident -->|"the contrast with the endpoints"| this
needs -->|"prerequisite"| ident
setup -->|"the real work"| needs
reduce -->|"do this after"| setup
pna -->|"before you start"| this
fail -->|"the symptom"| pna
fn -->|"set-once"| this
dbl -->|"redundancy"| this
sburi -->|"easy to get wrong"| dbl
kuri -->|"easy to get wrong"| dbl
db -->|"prerequisite"| this
tbl -->|"replacement cost"| db
subset -->|"what Terraform cannot express"| this
this -->|"creates"| res
res -->|"emits"| out
classDef me fill:#0078D4,stroke:#004578,color:#fff;
classDef keystone fill:#004578,stroke:#001f3f,color:#fff;
classDef sib fill:#eef2f7,stroke:#b8c4d0,color:#1b1b1b;
class this me;
class res keystone;
class what,ident,needs,setup,reduce,pna,fail,fn,dbl,sburi,kuri,db,tbl,subset,out sib;
Resource inventory
| Resource | Count | Notes |
|---|---|---|
azurerm_digital_twins_time_series_database_connection.this |
1 | The keystone. A child of the Digital Twins instance; supports no tags, and every field is force-new. |
| Requirement | Value |
|---|---|
| Terraform | >= 1.12.0 |
hashicorp/azurerm |
~> 4.0 |
| Provider block | None in this module. The caller configures provider "azurerm", including the mandatory features {} block, and supplies authentication. |
Schema notes that bite — confirmed against the live provider schema:
- Every field is force-new, including
eventhub_consumer_group_nameandkusto_table_name. The provider defines no update operation, which is why the module'stimeoutsobject has noupdatekey. eventhub_namespace_endpoint_urimust use thesb://protocol.https://is the intuitive guess and is rejected by the service.kusto_cluster_uriis the cluster'suri— not itsdata_ingestion_uri, and not its Resource ID.- The namespace and the cluster are each supplied twice, as a Resource ID and as a URI. Nothing checks that the two representations agree; a mismatch surfaces at apply.
- No output exists for the Event Hubs namespace endpoint URI. Neither the resource nor its data source exposes one, so it has to be composed from the namespace name.
- The Kusto database must already exist. Digital Twins creates the table inside it, along with its ingestion mapping.
- Changing
kusto_table_namereplaces the connection and leaves the old table behind, still holding the history recorded up to that point. It is not renamed and not removed. - The provider exposes a subset of the ARM API. The resource type also supports separate tables for twin-lifecycle and relationship-lifecycle events, a switch for recording property removals, and a user-assigned identity option — none of which is available here. Terraform can express only twin and relationship property history, authenticated by the system-assigned identity.
- No
tags. The universal tail istimeoutsonly, minusupdate.
Two audiences here, and conflating them is the most common source of trouble. The caller needs to create the resource; the Digital Twins instance's own identity needs to do the work.
The caller:
| Scope | Role / permission | Why |
|---|---|---|
| The Digital Twins instance | Microsoft.DigitalTwins/digitalTwinsInstances/timeSeriesDatabaseConnections/write — held by Azure Digital Twins Data Owner, or Contributor on the instance |
The connection is a child of the instance. |
The Digital Twins instance's system-assigned managed identity, during setup:
| Scope | Role | Why |
|---|---|---|
| The event hub | Azure Event Hubs Data Owner | Configuring the hub for history delivery. |
| The Kusto cluster or database | Contributor | Creating the history table. |
| The Kusto database (principal assignment) | Admin | Creating the table and its ingestion mapping. This is a Kusto database principal assignment, not an Azure RBAC role. |
💡 After setup these can be reduced. For ongoing operation the instance's identity needs only Azure Event Hubs Data Sender on the Event Hubs resource. Dropping Contributor and the database Admin assignment afterwards is the least-privilege end state — and it is the step that gets skipped, leaving a service identity with Contributor on an analytics cluster indefinitely.
- The
Microsoft.DigitalTwins,Microsoft.EventHubandMicrosoft.Kustoresource providers registered on the subscription. - A Digital Twins instance with a system-assigned managed identity already enabled. This module cannot enable it.
- An Event Hubs namespace with an event hub, preferably dedicated to data history, and preferably with a dedicated consumer group.
- A Kusto cluster and an existing database in it. Digital Twins creates the table, not the database.
- The three setup grants above, in place before this resource is created.
⚠️ The Kusto cluster must have public network access enabled while the connection is created. With it disabled, setup fails. Apply network restrictions afterwards, and switch the database's data connection to a system-assigned identity at the same time.
terraform-azurerm-digital-twins-time-series-database-connection/
├── providers.tf # required_version + the pinned azurerm provider. No provider block.
├── variables.tf # name, digital_twins_id, the Event Hub hop, the Kusto destination, timeouts
├── main.tf # the keystone azurerm_digital_twins_time_series_database_connection.this
├── outputs.tf # id, name, digital_twins_id, and the Kusto/Event Hub destination
├── README.md # this document
├── SCOPE.md # the cross-module contract
├── LICENSE # MIT
└── .gitignore
provider "azurerm" {
features {}
}
module "dt_history" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-time-series-database-connection.git?ref=v1.0.0"
name = "adt-history"
digital_twins_id = module.dt.id
eventhub_namespace_id = module.ehns.id
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net"
eventhub_name = module.ehns.eventhub_names["history"]
kusto_cluster_id = module.adx.id
kusto_cluster_uri = module.adx.uri
kusto_database_name = module.adx_db.name
}ℹ️ The caller configures the provider, its authentication, and the mandatory
features {}block. This module declares none of them.
⚠️ Note what is not here: no keys, no connection strings. The instance's managed identity does the work — which is why the grants in the permissions section are the real prerequisite, not an afterthought.
Consumes
| Input | Type | Source module |
|---|---|---|
digital_twins_id |
string |
terraform-azurerm-digital-twins-instance → id |
eventhub_namespace_id |
string |
terraform-azurerm-eventhub-namespace → id |
eventhub_namespace_endpoint_uri |
string |
composed — sb://<namespace name>.servicebus.windows.net |
eventhub_name |
string |
terraform-azurerm-eventhub-namespace → eventhub_names |
eventhub_consumer_group_name |
string |
a consumer group on that namespace |
kusto_cluster_id |
string |
terraform-azurerm-kusto-cluster → id |
kusto_cluster_uri |
string |
terraform-azurerm-kusto-cluster → uri |
kusto_database_name |
string |
terraform-azurerm-kusto-database → name |
ℹ️
eventhub_namespace_endpoint_uriis the one input with no output to wire from — see example 3.
Emits
| Output | Description | Consumed by |
|---|---|---|
id |
The connection's Resource ID, nested beneath the instance. | imports, state reading |
name |
The connection name. Force-new. | review |
digital_twins_id |
The instance this connection belongs to. | review |
kusto_cluster_uri |
The cluster the history is written to. | analytics tooling, dashboards |
kusto_database_name |
The database holding the history table. | analytics tooling, dashboards |
kusto_table_name |
The history table, or null for the service default. | analytics tooling, dashboards |
eventhub_name |
The event hub history flows through. | runbooks, troubleshooting |
eventhub_consumer_group_name |
The consumer group ADX reads with, or null for the default. | runbooks, troubleshooting |
🔒 There are no sensitive outputs, because there are no secrets to withhold. That is worth stating explicitly given the other three modules in this family emit a dead-letter posture flag and carefully withhold credentials.
1 · The minimal call
module "dt_history" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-time-series-database-connection.git?ref=v1.0.0"
name = "adt-history"
digital_twins_id = module.dt.id
eventhub_namespace_id = module.ehns.id
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net"
eventhub_name = module.ehns.eventhub_names["history"]
kusto_cluster_id = module.adx.id
kusto_cluster_uri = module.adx.uri
kusto_database_name = module.adx_db.name
}💡 Eight inputs, no credentials. The table name and consumer group are left to their service defaults here — example 5 argues for setting the consumer group anyway.
2 · The instance needs a system-assigned identity first
module "dt" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-instance.git?ref=v1.0.0"
name = "adt-twins-prod"
resource_group_name = module.rg.name
location = module.rg.location
identity = {
type = "SystemAssigned" # required for data history
}
}
⚠️ Not optional and not something this module can arrange. Without an identity on the instance there is nothing to authenticate the connection, and the apply fails.
ℹ️ The provider's resource does not accept a user-assigned identity for this connection, even though the underlying API supports one. System-assigned is the only shape Terraform can express here.
3 · The `sb://` URI you have to compose
# ✅ Correct.
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net"# ❌ Refused at `terraform validate`.
eventhub_namespace_endpoint_uri = "https://${module.ehns.name}.servicebus.windows.net"Error: Invalid value for variable
eventhub_namespace_endpoint_uri must start with "sb://" - the service requires that
protocol, for example "sb://my-namespace.servicebus.windows.net". An "https://" URI is
the intuitive guess and is wrong. The provider itself checks only that the string is
non-empty, so this is caught here or not at all before apply.
ℹ️ There is genuinely no output to wire this from: neither
azurerm_eventhub_namespacenor its data source exposes an endpoint URI, so composition from the name is the supported approach rather than a workaround.
🔒 Every check on this argument is the module's. The provider carries
StringIsNotEmptyon it and nothing else — no protocol check, no shape check, no comparison againsteventhub_namespace_id. Anhttps://URI, a bare hostname, and the URI of an entirely different namespace all pass the provider's own validation untouched and fail at apply, after the create has begun doing real work.
💡 These fire at
terraform validate— offline, without credentials — rather than at plan.
4 · Two representations of the same two resources
# The Event Hubs namespace, twice:
eventhub_namespace_id = module.ehns.id # Resource ID
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net" # URI
# The Kusto cluster, twice:
kusto_cluster_id = module.adx.id # Resource ID
kusto_cluster_uri = module.adx.uri # URI
⚠️ Nothing validates that the ID and the URI in each pair refer to the same resource. Pass a URI from one cluster and an ID from another and the plan is clean; the failure arrives at apply, or worse, the connection is created pointing somewhere unintended.
💡 Deriving both from the same module reference — as above — makes the mismatch structurally impossible. Hard-coding either one is where this goes wrong.
5 · Give ADX its own consumer group
module "dt_history" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-time-series-database-connection.git?ref=v1.0.0"
name = "adt-history"
digital_twins_id = module.dt.id
eventhub_namespace_id = module.ehns.id
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net"
eventhub_name = module.ehns.eventhub_names["history"]
eventhub_consumer_group_name = "adx-history"
kusto_cluster_id = module.adx.id
kusto_cluster_uri = module.adx.uri
kusto_database_name = module.adx_db.name
}💡 A consumer group tracks its own read position. Sharing
$Defaultwith any other reader of the same event hub makes the two compete, and you can lose history records without anything reporting an error.
⚠️ Force-new, like everything else here. Decide it at creation.
6 · Naming the history table
module "dt_history" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-time-series-database-connection.git?ref=v1.0.0"
digital_twins_id = var.digital_twins_id
eventhub_name = "eventhub-example"
eventhub_namespace_endpoint_uri = "eventhub-namespace-endpoint-uri"
eventhub_namespace_id = var.eventhub_namespace_id
kusto_cluster_id = var.kusto_cluster_id
kusto_cluster_uri = "kusto-cluster-uri"
kusto_database_name = "kusto-database-example"
name = "dt-history-example"
kusto_table_name = "TwinPropertyHistory"
}ℹ️ Left unset, the service names it
AdtPropertyEvents. The module does not restate that default as its own, so the service keeps owning it — the output readsnullin that case, and the table still exists under the service's name.
⚠️ Force-new, and the replacement is not clean: changing it leaves the old table behind holding the history recorded up to that point. It is not renamed and not removed, so you end up with two tables and a gap in whichever one you query.
7 · The ordering trap — public network access
# 1 · Create the cluster reachable.
module "adx" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster.git?ref=v1.0.0"
name = "adxtwinsprod"
resource_group_name = module.rg.name
location = module.rg.location
sku = {
name = "Standard_D13_v2"
capacity = 2
}
public_network_access_enabled = true # required WHILE the connection is created
}
# 2 · Create the connection (this module).
# 3 · THEN restrict the network, and switch the database's data connection to a
# system-assigned identity so the pieces can still reach each other.
⚠️ With public network access disabled, setup fails — Digital Twins cannot reach in to create the table and its ingestion mapping. The error is an internal-server-error that does not name the cause, which makes it an expensive thing to debug from first principles.
💡 This inverts the usual instinct of locking a resource down before wiring anything to it. Sequence it deliberately, and note that step 3 is a separate change that no single apply will do for you.
8 · No update operation at all
timeouts = {
create = "60m"
read = "5m"
delete = "30m"
# no `update` — the object does not accept one
}ℹ️ Every field on this resource is force-new, so the provider defines no update. The
timeoutsobject omits the key rather than accepting and ignoring it, which turns a misunderstanding into a type error at plan time.
💡 Raise
createif the cluster is busy. Creation is not a metadata write: Digital Twins builds the table and its ingestion mapping inside this timeout.
9 · Reducing the grants after setup
# DURING setup — all three needed by the INSTANCE's identity:
# Azure Event Hubs Data Owner on the event hub
# Contributor on the Kusto cluster or database
# database principal Admin on the Kusto database
# AFTER setup — this is sufficient for ongoing operation:
module "dt_history_roles" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"
scope = module.ehns.id
role_assignments = {
eventhub_sender = {
scope = module.ehns.id
role_definition_name = "Azure Event Hubs Data Sender"
principal_id = module.dt.identity_principal_id
}
}
}💡 The reduction is the part that gets skipped, and skipping it leaves a service identity holding Contributor on an analytics cluster forever. Data history itself needs only Data Sender once the table exists.
⚠️ Do not reduce before the connection is created — the setup grants are what let Digital Twins build the table. And if you later replace the connection, the setup grants are needed again.
10 · What Terraform cannot express here
Supported by the ARM API, NOT exposed by the pinned provider:
adxRelationshipLifecycleEventsTableName relationship lifecycle events -> its own table
adxTwinLifecycleEventsTableName twin lifecycle events -> its own table
recordPropertyAndItemRemovals record property/item DELETIONS too
identity user-assigned identity option
ℹ️ So a Terraform-managed connection historizes twin and relationship property updates, authenticated by the system-assigned identity, and nothing else. Lifecycle events — twins and relationships being created and deleted — are not recorded.
⚠️ Worth knowing before promising an audit trail. "We have data history" and "we can show when this twin was deleted" are different claims, and only the first one is true of what this module builds.
11 · Querying what it produces
output "history_query_target" {
value = {
cluster = module.dt_history.kusto_cluster_uri
database = module.dt_history.kusto_database_name
table = module.dt_history.kusto_table_name # null = AdtPropertyEvents
}
}AdtPropertyEvents
| where TimeStamp > ago(1h)
| where Id == "thermostat-42"
| project TimeStamp, Key, Value💡 The destination is emitted in three pieces because that is exactly what a query needs. Reassembling it from a Resource ID is avoidable friction, and the table name in particular is not derivable from anything else.
ℹ️ Default ingestion latency is roughly ten minutes. If that is too slow, enable streaming ingestion on the cluster — a cluster-side setting, not something this connection controls.
12 · Destroying the connection keeps the data
# module.dt_history.azurerm_digital_twins_time_series_database_connection.this will be destroyed
ℹ️ The table and everything in it survive. Only the connection goes, so history stops accumulating and what has already been recorded stays queryable.
💡 That makes this a safe resource to replace — unlike most of the destructive plans in this family. The one cost is the gap: nothing is recorded between the destroy and the next create.
13 · The name constraint, and why it is not one regex
name = "adt-history" # 3-50 chars, letters/digits/hyphens, not all digits, no leading or trailing hyphen# ❌ All refused at `terraform validate`:
name = "ab" # too short - the floor is 3
name = "123" # all digits
name = "-history" # leading hyphen
name = "history-" # trailing hyphen
name = "adt_history" # underscore
⚠️ An earlier version of this module encoded the range as 2–49, and that was wrong in both directions. The provider checkslen < 3andlen > 50as two separate steps, so the legal range is 3 to 50. The old range refused a legal 50-character name — and because a failingvalidation {}blocksterraform destroyas well as apply, a connection already created with such a name could not have been destroyed through the module. Being stricter than the provider is the one direction a module must not take.
ℹ️ The provider expresses the rest as
^[A-Za-z0-9][A-Za-z0-9-]+[A-Za-z0-9]$plus a dedicated all-digits check. The service documents it with lookaheads, which Terraform's Go RE2 engine does not support — that pattern fails to compile rather than matching — so the module carries the length, character-set and all-digits rules as three separate checks.
💡 Worth knowing generally: a service's own regex cannot always be pasted into a
validationblock, and a lookahead that fails to compile is a hard error at evaluation time rather than a permissive fallback.
14 · 🏗️ End-to-end composition
provider "azurerm" {
features {}
}
module "rg" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"
name = "rg-twins-prod"
location = "eastus"
}
# 1 · The twin graph, WITH a system-assigned identity — the whole basis of this connection.
module "dt" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-instance.git?ref=v1.0.0"
name = "adt-twins-prod"
resource_group_name = module.rg.name
location = module.rg.location
identity = {
type = "SystemAssigned"
}
}
# 2 · The Event Hub hop, with a dedicated hub and consumer group for history.
module "ehns" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-eventhub-namespace.git?ref=v1.0.0"
name = "ehns-twins-prod"
resource_group_name = module.rg.name
location = module.rg.location
sku = "Standard"
event_hubs = {
history = {
name = "history"
partition_count = 4
message_retention = 7
}
}
consumer_groups = {
adx = {
name = "adx-history"
eventhub_key = "history"
}
}
}
# 3 · The analytics destination. Reachable during setup — see example 7.
module "adx" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster.git?ref=v1.0.0"
name = "adxtwinsprod"
resource_group_name = module.rg.name
location = module.rg.location
sku = {
name = "Standard_D13_v2"
capacity = 2
}
public_network_access_enabled = true
}
module "adx_db" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-database.git?ref=v1.0.0"
name = "twins-history"
resource_group_name = module.rg.name
location = module.rg.location
cluster_name = module.adx.name
}
# 4 · The setup grants, to the INSTANCE's identity — not the caller's.
module "dt_history_setup_roles" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"
scope = module.ehns.id
role_assignments = {
eventhub_owner = {
scope = module.ehns.id
role_definition_name = "Azure Event Hubs Data Owner"
principal_id = module.dt.identity_principal_id
}
kusto_contributor = {
scope = module.adx.id
role_definition_name = "Contributor"
principal_id = module.dt.identity_principal_id
}
}
}
# The Kusto database principal assignment with role Admin is a distinct resource type with no module
# in this suite yet, so declare it directly.
resource "azurerm_kusto_database_principal_assignment" "dt" {
name = "dt-history-admin"
resource_group_name = module.rg.name
cluster_name = module.adx.name
database_name = module.adx_db.name
tenant_id = module.dt.identity_tenant_id
principal_id = module.dt.identity_principal_id
principal_type = "App"
role = "Admin"
}
# 5 · The connection. This is the module.
module "dt_history" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-digital-twins-time-series-database-connection.git?ref=v1.0.0"
name = "adt-history"
digital_twins_id = module.dt.id
eventhub_namespace_id = module.ehns.id
eventhub_namespace_endpoint_uri = "sb://${module.ehns.name}.servicebus.windows.net"
eventhub_name = module.ehns.eventhub_names["history"]
eventhub_consumer_group_name = "adx-history"
kusto_cluster_id = module.adx.id
kusto_cluster_uri = module.adx.uri
kusto_database_name = module.adx_db.name
timeouts = {
create = "60m"
}
# Nothing in the arguments references the grants, so the ordering must be explicit.
depends_on = [
module.dt_history_setup_roles,
azurerm_kusto_database_principal_assignment.dt,
]
}
output "twins_history" {
value = {
cluster = module.dt_history.kusto_cluster_uri
database = module.dt_history.kusto_database_name
table = module.dt_history.kusto_table_name # null = AdtPropertyEvents
}
}
⚠️ Two steps deliberately remain after this applies: reduce the setup grants to Event Hubs Data Sender (example 9), and restrict the Kusto cluster's network (example 7). Neither can be part of the same apply — the connection needs the wider access while it is being created.
🔒 What the composition gets right: no credential appears anywhere, the identity is system-assigned rather than a secret, the event hub and consumer group are dedicated to history so nothing competes for the read position, and both URI/ID pairs derive from the same module reference so they cannot disagree.
💡 The
depends_onis not decoration. Nothing in this module's arguments references the role assignments, so without it Terraform is free to create the connection before the grants exist — and role assignments take time to propagate even once created.
| Input | Type | Default | Notes |
|---|---|---|---|
name |
string |
— | Required. Force-new. Validated: 3–50 chars, letters/digits/hyphens, not all digits, beginning and ending with a letter or digit. |
digital_twins_id |
string |
— | Required. Force-new. The instance needs a system-assigned identity. |
eventhub_namespace_id |
string |
— | Required. Force-new. |
eventhub_namespace_endpoint_uri |
string |
— | Required. Force-new. Validated: must start sb://. |
eventhub_name |
string |
— | Required. Force-new. |
eventhub_consumer_group_name |
string |
null |
Force-new. Null = $Default; prefer a dedicated group. |
kusto_cluster_id |
string |
— | Required. Force-new. |
kusto_cluster_uri |
string |
— | Required. Force-new. Validated: must start https://. |
kusto_database_name |
string |
— | Required. Force-new. Must already exist. |
kusto_table_name |
string |
null |
Force-new. Null = AdtPropertyEvents. |
timeouts |
object(...) |
null |
create / read / delete — no update. |
Full schemas
variable "name" {
type = string
# Validated as separate checks rather than the service's own pattern: Terraform's regex engine
# (Go RE2) has no lookahead, so `^(?![0-9]+$)(?!-)...` cannot be used — it fails to compile.
# 3-50 chars, letters/digits/hyphens, not all digits, no leading or trailing hyphen.
}
variable "digital_twins_id" {
type = string
# ⚠️ The instance must already have a SYSTEM-ASSIGNED managed identity — this connection has no
# other way to authenticate. Enable it on the instance module, not here.
}
variable "eventhub_namespace_id" {
type = string
# ℹ️ The service wants the namespace as BOTH a Resource ID and an sb:// URI. They must agree;
# nothing checks that they do.
}
variable "eventhub_namespace_endpoint_uri" {
type = string
# Validated: must start with "sb://". No module output exists — compose it:
# "sb://${module.ehns.name}.servicebus.windows.net"
}
variable "eventhub_name" {
type = string
# 💡 Dedicate an event hub to history rather than sharing one with another consumer.
}
variable "eventhub_consumer_group_name" {
type = string
default = null # service default: $Default
# 💡 Supply a dedicated group. Sharing $Default with another reader makes the two compete for the
# read position and can cost you history records.
}
variable "kusto_cluster_id" {
type = string
}
variable "kusto_cluster_uri" {
type = string
# Validated: must start with "https://". This is the cluster's `uri`, not `data_ingestion_uri`.
}
variable "kusto_database_name" {
type = string
# ⚠️ The database must already exist. Digital Twins creates the TABLE inside it.
}
variable "kusto_table_name" {
type = string
default = null # service default: AdtPropertyEvents
# ⚠️ Force-new, and the old table is LEFT BEHIND holding the history recorded up to that point.
}
variable "timeouts" {
type = object({
create = optional(string)
read = optional(string)
delete = optional(string)
})
default = null
# ⚠️ No `update` key, and that is not an omission: every field is force-new, so the provider
# defines no update operation.
}| Output | Description | Sensitive |
|---|---|---|
id |
The connection's Resource ID, nested beneath the instance. | no |
name |
The connection name. Force-new, like all ten arguments. | no |
digital_twins_id |
The instance this connection belongs to — and whose system-assigned identity is the only authentication path here. | no |
kusto_cluster_uri |
The cluster the history is written to. The QUERY endpoint, not the ingest- one. |
no |
kusto_cluster_id |
The cluster's Resource ID, read back in canonical case. Use it to scope the cluster-side role assignments. | no |
kusto_database_name |
The database holding the history table. Must already exist. | no |
kusto_table_name |
The history table. Always populated, never null — the provider's schema defaults it to AdtPropertyEvents and reads that value back. |
no |
kusto_query_reference |
Map of cluster URI, database and table — the three values a Kusto query needs, assembled. | no |
eventhub_name |
The event hub history flows through. Its retention is the history's blast radius. | no |
eventhub_namespace_id |
The namespace containing it — emitted to scope the namespace-side role assignment. | no |
eventhub_consumer_group_name |
The consumer group ADX reads with. Always populated, never null — the schema defaults it to $Default. |
no |
table_is_the_service_default |
Whether the table name was inherited rather than chosen. Derived, so known at plan time — the only place the distinction survives. | no |
consumer_group_is_the_shared_default |
Whether ADX reads with the shared $Default. True is the case to review: a competing reader costs history records, with no error. |
no |
destroy_leaves_the_kusto_table_behind |
Always true. The provider never sends cleanupConnectionArtifacts, so a destroy leaves the ADX table, its mapping and the data connection in place — stored, billed, unwritten. |
no |
grants_no_access_by_itself |
Always true. The instance identity's roles on the Event Hub and the cluster are the real work, and the failure mode is silence, not an error. |
no |
requires_public_kusto_access_during_setup |
Always true. Restrict the cluster's network after this applies, not before. |
no |
every_argument_is_force_new |
Always true — all ten, which is why the timeouts type declares no update key. |
no |
the_two_representations_are_never_compared_by_the_provider |
Always true. Four arguments describe two resources and the provider validates each in isolation; the name comparison is this module's. |
no |
🔒 Nothing here is sensitive, because this resource accepts and produces no secrets. That is the substantive difference between it and the three endpoint modules in this family, and the reason to prefer data history over a hand-rolled export pipeline where either would do.
-
The identity-based design is why this module is the pleasant one. Its three siblings each hold two live credentials that reach Terraform state and cannot be avoided. This one holds none: the instance's system-assigned identity authenticates, so the security question becomes "which roles, for how long" rather than "where do these keys end up". Where a design choice exists between an endpoint and data history, that difference is worth weighing.
-
The grants are the real work, and they are reducible. Three grants to the instance's identity are needed during setup — Event Hubs Data Owner, Kusto Contributor, and a database principal assignment with role Admin — because Digital Twins builds the table on your behalf. Afterwards, Event Hubs Data Sender alone suffices. Leaving Contributor in place is the default outcome of not thinking about it, and the wrong end state.
-
The public-network-access ordering inverts the usual instinct. A locked-down cluster cannot be set up, because Digital Twins reaches in to create the table and mapping. So: open, connect, then restrict — and the restriction is a separate change, not part of the same apply. The failure mode if you get this backwards is an internal-server-error that names nothing.
-
Four inputs describe two resources, and THE PROVIDER NEVER COMPARES THEM. The Event Hubs namespace and the Kusto cluster are each supplied as both a Resource ID and a URI. The provider validates each in isolation — the two IDs through their parsers, the two URIs through a bare non-emptiness check — so a configuration naming one namespace by ID and a different one by URI passes its validation entirely and fails at apply. This module compares the NAME in each pair: the last segment of the ID against the first label of the URI host, lowercased. Only the first label is compared, so sovereign-cloud hostnames (
servicebus.chinacloudapi.cn,kusto.usgovcloudapi.net) stay legal. Deriving both from the same module reference still makes a mismatch structurally impossible, and remains the better habit. -
A destroy leaves the Azure Data Explorer table behind. The service's delete accepts a
cleanupConnectionArtifactsoption and the provider never sets it, soterraform destroyremoves the connection record and leaves the table, its ingestion mapping and the cluster-side data connection in place — still stored, still billed, no longer written to. There is no argument to change that, and the same is true of any replacement, since every argument is force-new. Cleaning up is a manual step in Kusto. -
Two arguments read back as their defaults rather than as null.
kusto_table_nameandeventhub_consumer_group_namecarry schema defaults (AdtPropertyEventsand$Default), so leaving them null still sends those values and the attributes are never null.table_is_the_service_defaultandconsumer_group_is_the_shared_defaultare derived from the inputs, which is the only place the distinction survives. -
You cannot type
$Default. The provider's consumer-group validator is^[a-zA-Z0-9]([-._a-zA-Z0-9]{0,48}[a-zA-Z0-9])?$, which permits no dollar sign — so settingeventhub_consumer_group_nameto the exact value the provider sends when you omit it is refused atterraform validate. The schema's own default is not a legal input to the schema. Omit the argument to get it. Note also that the consumer-group limit is 50 characters while the event hub's is 256, despite near-identical error messages. -
One provider error names the wrong argument. When a refresh cannot parse the cluster ID returned by Azure, the message reads
parsing `kusto_cluster_uri`— the value it could not parse iskusto_cluster_id. And the Kusto database-name validator prints a 4-to-22-character limit while enforcing 260. -
The service's own name regex cannot be used directly. It relies on negative lookahead, which Go RE2 does not support — the pattern fails to compile rather than matching permissively. The same rules are enforced as separate checks. This generalizes: a documented service regex is not automatically a usable
validationcondition. -
Everything is force-new, so there is nothing to update. The
timeoutsobject omitsupdaterather than accepting and ignoring it, which turns a misunderstanding into a plan-time type error. The practical consequence is that this is a set-once configuration — including the table name and consumer group, both of which look incidental and are not. -
Replacement is unusually cheap here, with one asterisk. Destroying the connection leaves the table and its data intact, so history stops accumulating rather than disappearing. The asterisk is
kusto_table_name: change it and the old table stays behind holding the history so far, leaving two tables and a gap in each. -
The provider covers less than the API. Twin-lifecycle and relationship-lifecycle event tables, property-removal recording, and user-assigned identity all exist in the ARM API and none is exposed here. So a Terraform-managed connection records property updates only — relevant if anyone is treating data history as an audit trail for creations and deletions.
-
No
tags. The resource type does not support them. Tag the instance, the namespace and the cluster instead.
| Concern | Secure default (empty call) | Opt-out (caller must type it) |
|---|---|---|
| Credentials | none accepted and none emitted — authentication is a managed identity | — |
| Identity | the instance's system-assigned identity, documented as a hard prerequisite | — |
| Grant duration | the setup grants are documented as reducible, with the end state named | leave Contributor in place, knowingly |
| Network posture | the required order is documented — open during setup, restrict afterwards | — |
| URI correctness | sb:// and https:// prefixes validated, because both fail only at apply |
— |
| Name correctness | the service's constraint enforced without relying on an unsupported regex | — |
| Service defaults | left to the service (AdtPropertyEvents, $Default) rather than restated as module defaults |
set them explicitly, which is recommended for the consumer group |
| Force-new fields | update omitted from timeouts so the set-once nature is a type error, not a surprise |
— |
- Before applying: confirm the instance has a system-assigned identity, and that all three setup grants are in place.
- Before applying: confirm the Kusto cluster is reachable and the database already exists.
- Before applying: confirm each ID/URI pair derives from the same resource.
- After applying: reduce the grants to Event Hubs Data Sender, and restrict the cluster's network. Both are separate changes.
terraform init -backend=false
terraform validate
terraform fmt -check- Pin the source to a tag —
?ref=v1.0.0— never a branch. - Plan-only from here. A human applies from CI.
⚠️ Order on first apply: identity, then grants, then this connection. Usedepends_on— nothing in the arguments expresses it.⚠️ Raisecreatefor a busy cluster; table creation happens inside that timeout.⚠️ Two follow-up changes after a successful apply: reduce the grants, restrict the network.- ℹ️ Destroying this is comparatively safe — the table and its data remain.
terraform validate and terraform fmt -check are the offline gate. They confirm:
- all eight required inputs are present and typed
string; namesatisfies the length, character, all-digits and hyphen rules;eventhub_namespace_endpoint_uristarts withsb://;kusto_cluster_uristarts withhttps://;- the
timeoutsvalue is an object of Go duration strings, with noupdatekey because the provider defines no update operation - and note that passing one anyway is SILENTLY DISCARDED rather than refused, producing no error atvalidate, none atplan, and no trace in the resulting value; - the module declares no
providerblock.
What only plan and apply exercise:
- whether the instance actually has a system-assigned identity;
- whether that identity holds all three setup grants, and whether they have propagated;
- whether the Kusto database exists;
- whether the cluster permits public network access at creation time;
- whether each ID/URI pair genuinely refers to the same resource;
- whether table creation completes inside the
createtimeout.
Note the name validation was verified by evaluating the condition directly rather than trusting terraform validate, which does not evaluate module-input validations — a lookahead-based pattern would have shipped clean and failed in a caller's plan.
Outputs:
digital_twins_id = "/subscriptions/00000000-.../providers/Microsoft.DigitalTwins/digitalTwinsInstances/adt-twins-prod"
eventhub_consumer_group_name = "adx-history"
eventhub_name = "history"
id = "/subscriptions/00000000-.../digitalTwinsInstances/adt-twins-prod/timeSeriesDatabaseConnections/adt-history"
kusto_cluster_uri = "https://adxtwinsprod.eastus.kusto.windows.net"
kusto_database_name = "twins-history"
kusto_table_name = null
name = "adt-history"
ℹ️
kusto_table_name = nullmeans the service default is in place — the table exists asAdtPropertyEvents. The module does not restate that default as its own, so the null is accurate rather than missing.
| Symptom | Cause | Fix |
|---|---|---|
Plan rejects eventhub_namespace_endpoint_uri |
An https:// URI was passed. |
Use sb:// (example 3). |
Plan rejects kusto_cluster_uri |
A Resource ID or the data_ingestion_uri was passed. |
Use the cluster's uri output. |
Plan rejects name |
Too short, all digits, a leading/trailing hyphen, or an underscore. | See example 13. |
Plan rejects an update timeout |
Every field is force-new, so no update operation exists. | Remove the key (example 8). |
| Apply fails: "the resource could not ACT due to an internal server error" | The Kusto cluster has public network access disabled. | Enable it for setup, restrict afterwards (example 7). |
| Apply fails with a permission error on the database | The instance's identity lacks the database principal Admin assignment. | Add it, wait for propagation, retry (example 14). |
| Apply fails reaching the event hub | The identity lacks Azure Event Hubs Data Owner during setup. | Grant it; Data Sender alone is not enough to create the connection. |
| Apply fails and the instance has no identity | Data history needs a system-assigned identity. | Enable it on the instance module (example 2). |
| Apply times out during creation | Table and mapping creation exceeded create. |
Raise create; 60m is a reasonable starting point. |
| Apply succeeds but no rows appear | Default ingestion latency is around ten minutes. | Wait, then check. Enable streaming ingestion on the cluster if that is too slow. |
| Rows stopped appearing, no errors | Another reader is sharing the $Default consumer group. |
Give ADX its own consumer group (example 5). |
| Two history tables exist, each with a gap | kusto_table_name was changed; the old table was left behind. |
Expected. Query both, or backfill. |
| Twin deletions are not in the history | The provider exposes no lifecycle-event tables. | Not available through Terraform (example 10). |
| The identity still holds Contributor months later | The post-setup reduction was skipped. | Reduce to Event Hubs Data Sender (example 9). |
azurerm_digital_twins_time_series_database_connection— provider documentation for this resource.azurerm_digital_twins_instance— the instance and its managed identity.azurerm_kusto_database_principal_assignment— the database Admin assignment the setup needs.- Azure Digital Twins data history — the required roles, the reduction to Data Sender, and the ingestion-latency figures.
- Create a data history connection — the public-network-access requirement and the correct order for applying network restrictions.
- Sibling modules in this family:
terraform-azurerm-digital-twins-instance,terraform-azurerm-digital-twins-endpoint-eventgrid,terraform-azurerm-digital-twins-endpoint-eventhub,terraform-azurerm-digital-twins-endpoint-servicebus. - This module's
SCOPE.md.
💙 "Infrastructure as Code should be standardized, consistent, and secure."