☁️ Azure Stream Analytics IoT Hub Stream Input Terraform Module

One IoT Hub stream input on a Stream Analytics job — where the job's query reads device telemetry from an IoT Hub's built-in endpoint — targeting hashicorp/azurerm ~> 4.0.

🧩 Overview

📡 Declares where a Stream Analytics job reads device telemetry from — IoT Hub, built-in endpoint, consumer group, serialization.
🔴 States the limitation that defines it: the credential is required and this resource offers no managed-identity option at all, so a secret is in state unconditionally.
🔑 Records that shared_access_policy_key is FORCE-NEW — the only credential in this provider family that is — so rotating the key destroys and recreates the input.
⚠️ Records that the force-new marking does not catch an out-of-band rotation, which is the easy thing to conflate.
🕳️ Records that the importer performs no datasource type check, so importing the wrong kind of input succeeds silently and never converges.
✅ Notes the one respect in which this resource is stricter than its siblings: the consumer group is required, so there is no accidental fall-through to $Default.
🏷️ Carries no tags and no location; the universal tail is timeouts only.

💡 Why it matters: this suite's secure-by-default rule has nothing to work with here. The provider makes both credential fields required and gives no identity alternative, so there is no safe empty call and no risky option to opt into. What a module can still do is refuse to pretend — validate the values, never emit the key, name the safer policy choice, and say plainly in the permissions table that whoever can plan can read a credential to the hub.

❤️ Support this project

If this module saved you time:

⭐ Star the repository — it helps other people find it.
💼 Connect on LinkedIn — linkedin.com/in/microsoftexpert
☕ Buy me a coffee — buymeacoffee.com/microsoftexpert

🗺️ Where this fits in the family

flowchart TB
  RG["terraform-azurerm-resource-group"]
  JOB["terraform-azurerm-stream-analytics-job"]
  HUB["terraform-azurerm-iothub"]
  KV["terraform-azurerm-key-vault"]
  THIS["terraform-azurerm-stream-analytics-stream-input-iothub"]
  EH["terraform-azurerm-stream-analytics-stream-input-eventhub"]
  QUERY["the job's transformation query, which reads FROM it by name"]

  RG -->|"name"| JOB
  JOB -->|"name plus resource_group_name"| THIS
  HUB -->|"name, endpoint, consumer group and a REQUIRED policy key"| THIS
  KV -.->|"read the key from here rather than committing it"| THIS
  JOB -->|"name plus resource_group_name"| EH
  EH -.->|"an input that DOES offer managed identity, unlike this one"| THIS
  THIS -.->|"is read from, but does not manage"| QUERY

  classDef me fill:#0078D4,stroke:#004578,stroke-width:2px,color:#ffffff
  classDef target fill:#004578,stroke:#00243d,stroke-width:2px,color:#ffffff
  classDef sib fill:#eef3f8,stroke:#9db4c9,color:#1a2733

  class THIS me
  class JOB target
  class RG,HUB,KV,EH,QUERY sib

The dotted edges carry the design advice. Read the key from Key Vault rather than committing it — the provider gives no way to omit it — and note that the event hub stream input does offer managed identity, so a pipeline that must avoid stored keys reaches Stream Analytics a different way rather than configuring this module differently.

🧬 What this module builds

flowchart TB
  IDENT["name, job name, resource_group_name"]
  SRC["iothub_namespace, endpoint, consumer group REQUIRED"]
  CRED["policy name and key, BOTH REQUIRED, key is FORCE-NEW"]
  SER["serialization: Avro Csv or Json"]
  THIS["azurerm_stream_analytics_stream_input_iothub.this"]
  OID["id"]
  OSEC["a_secret_is_in_state_unconditionally"]
  OROT["rotating_the_key_replaces_the_input"]
  OIMP["the_importer_does_not_check_the_datasource_type"]

  IDENT --> THIS
  SRC --> THIS
  CRED --> THIS
  SER --> THIS
  THIS --> OID
  THIS --> OSEC
  THIS --> OROT
  THIS --> OIMP

  classDef me fill:#0078D4,stroke:#004578,stroke-width:2px,color:#ffffff
  classDef target fill:#004578,stroke:#00243d,stroke-width:2px,color:#ffffff
  classDef sib fill:#eef3f8,stroke:#9db4c9,color:#1a2733

  class THIS me
  class OID target
  class IDENT,SRC,CRED,SER,OSEC,OROT,OIMP sib

Resource inventory

Resource	Count	Notes
`azurerm_stream_analytics_stream_input_iothub`	1 (`this`)	one stream input on one job

/subscriptions/SUB/resourceGroups/RG/providers/Microsoft.StreamAnalytics/streamingJobs/JOB/inputs/NAME

ℹ️ That ID identifies the input record on the job, not the IoT Hub. The hub has its own Resource ID under Microsoft.Devices.

✅ Provider / Versions

Requirement	Value
Terraform	`>= 1.12.0`
`hashicorp/azurerm`	`~> 4.0`
Provider block	None in this module — the caller configures the provider, its authentication, and the mandatory `features {}` block
Resources created	1

Schema notes that bite

🔴 The credential is REQUIRED and there is no authentication_mode argument. Both shared_access_policy_key and shared_access_policy_name are Required, and managed identity is not merely un-defaulted here — it cannot be selected.
🔴 shared_access_policy_key is FORCE-NEW, which no other credential in this provider family is. Changing it destroys and recreates the stream input. Because a Stream Analytics job must be stopped before its inputs can change, a key rotation here is a planned operational event rather than a quiet edit. The policy name beside it is not force-new.
⚠️ The force-new marking does not catch an out-of-band rotation, and the two are easy to conflate. It fires on a change Terraform can see in configuration; a key regenerated in the portal is invisible, because Azure never returns it.
🔴 The importer performs NO datasource type check. It validates only that the string parses as an Input Resource ID, so importing a blob or event hub input at this address succeeds silently — and every subsequent plan then wants to rewrite the datasource, a change that never converges. The blob reference input, the Service Bus outputs and the event hub V2 input all do check.
✅ eventhub_consumer_group_name is REQUIRED here and Optional on both event hub inputs. There is no accidental fall-through to $Default — the one respect in which this resource is stricter than its siblings.
⚠️ endpoint is validated by nothing beyond non-empty. In practice it is messages/events, the built-in Event Hub compatible endpoint; the other documented value is the operations-monitoring feed.
⚠️ iothub_namespace names the IoT Hub itself. A hub is a top-level resource with no namespace above it; the argument mirrors the ARM property name.
🔴 The serialization type pairing is enforced in one direction only on an input: a missing required field is fatal at apply (Csv needs encoding + field_delimiter, Json needs encoding), a surplus one is accepted and dropped in silence. The output resources close both directions.
🔴 Azure never returns the policy key on a read, so a key regenerated in the portal produces no drift.
✅ All four timeouts fields are honoured. Note a key rotation exercises delete + create, not update, because the key is force-new.
Force-new: name, stream_analytics_job_name, resource_group_name and shared_access_policy_key — four, where the siblings have three.

🔑 Required Azure RBAC Roles / Permissions

Least-privilege, at the smallest scope that works:

Microsoft.StreamAnalytics/streamingJobs/inputs/write, .../read and .../delete on the target job. A custom role scoped to the job is enough; Contributor is broader than necessary.
Microsoft.Devices/iotHubs/listkeys/action on the IoT Hub, for whatever reads the policy key to pass in. Because the key is required, this permission is effectively mandatory here.
No data-plane role is needed by the job, because it authenticates with the policy key rather than with an identity. That is not an advantage: it means the narrow, scopeable grant a managed identity would have used is unavailable.

🔒 Plan access IS credential access on this resource, unavoidably. The key is required, so it is in state for every configuration of this module — and if the policy is iothubowner, whoever can read that state can administer the hub and write its device registry. Settle that before granting plan rights.

Azure Prerequisites

An existing Stream Analytics job, in the resource group named here.
An existing IoT Hub, its built-in endpoint, a consumer group on that endpoint, and a shared access policy with service connect rights.
A transformation query on the job that reads FROM this input by name.
The caller configures provider "azurerm" { features {} }, authentication, and the subscription.

📁 Module Structure

terraform-azurerm-stream-analytics-stream-input-iothub/
├── providers.tf     # required_version, pinned azurerm ~> 4.0, no provider block
├── variables.tf     # 10 typed inputs, 18 validations, the universal timeouts tail
├── main.tf          # the keystone `this`, a static serialization block, dynamic timeouts
├── outputs.tf       # 38 outputs: id first, then posture, rotation and behaviour flags
├── README.md        # this file
├── SCOPE.md         # the cross-module contract
├── LICENSE          # MIT
└── .gitignore

⚙️ Quick Start

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

🔴 There is no key-free form of this call. a_secret_is_in_state_unconditionally is true — read the key from Key Vault so it is not in the repository, and put the state in an encrypted, access-controlled backend, because sensitive = true redacts the plan and does not encrypt state.

⚠️ The caller configures the provider, including the mandatory features {} block.

🔌 Cross-Module Contract

Consumes

Input	Type	Source
`stream_analytics_job_name`	`string`	`terraform-azurerm-stream-analytics-job` → `name`
`resource_group_name`	`string`	`terraform-azurerm-resource-group` → `name`
`iothub_namespace`	`string`	`terraform-azurerm-iothub` → `name`
`shared_access_policy_key`	`string` (sensitive)	a Key Vault secret — required, not omittable

Emits

Output	Description
`id`	Resource ID of the input record (first)
`name`	The identifier the query reads FROM
`stream_analytics_job_id`	The parent job's Resource ID, for RBAC scoping
`a_secret_is_in_state_unconditionally`	Constant `true`
`rotating_the_key_replaces_the_input`	Constant `true`
`the_importer_does_not_check_the_datasource_type`	Constant `true`

📚 Example Library

1 · The smallest real call

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

💡 Every argument above is required. There is no optional credential, no authentication_mode, and no default consumer group — this resource asks the caller to state everything.

2 · Reading the key from Key Vault

module "vault" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-key-vault.git?ref=v1.0.0"

  name                = "kv-streaming-01"
  resource_group_name = module.rg.name
  location            = module.rg.location
  tenant_id           = data.azurerm_client_config.current.tenant_id
}

data "azurerm_key_vault_secret" "iot_service_key" {
  name         = "iot-service-policy-key"
  key_vault_id = module.vault.id
}

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

🔒 This keeps the key out of the repository. It does not keep it out of state — sensitive_redacts_the_plan_and_does_not_encrypt_state is true, and the value lands in the state file in clear.

🔴 plan_access_is_credential_access is true. Because the key cannot be omitted, granting someone plan rights on a configuration containing this module is granting them that key.

3 · Choosing the narrow policy

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  # "service" carries service connect. "iothubowner" additionally carries
  # registry write and can administer the hub.
  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

output "over_privileged" {
  value = module.telemetry_input.uses_the_hub_owner_policy
  # false
}

🔒 an_iothub_policy_key_grants_what_the_policy_grants is true. Nothing in Azure narrows a key below the rights its policy carries, so the policy choice is the blast radius. Reading telemetry needs only service connect.

⚠️ uses_the_hub_owner_policy is reported, not refused: the provider accepts any policy name, and this module cannot see what rights a policy actually carries — the name is strong evidence, not proof.

4 · Rotating the key replaces the input

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

output "rotation_is_a_replacement" {
  value = module.telemetry_input.rotating_the_key_replaces_the_input
  # true
}

🔴 shared_access_policy_key is force-new — the only credential in this provider family that is. Change the Key Vault secret and the next plan shows destroy and create, not an update. Because a Stream Analytics job must be stopped before its inputs can change, that is a planned operational event.

💡 The policy name beside it is not force-new, so switching from iothubowner to service while keeping the same key updates in place. force_new_fields lists all four.

5 · The rotation force-new does not catch

output "which_key_is_deployed" {
  value = module.telemetry_input.shared_access_policy_key_fingerprint
  # "a251c2d1..." — changes when, and only when, the configured key changes
}

🔴 the_force_new_marking_does_not_catch_an_out_of_band_rotation is true, and this is the pair of facts most easily conflated. Force-new fires on a change Terraform can see in configuration. A key regenerated in the Azure portal is not such a change: Azure never returns the key, so the provider has nothing to compare, the plan is empty, and the job simply stops reading.

💡 Rotate by updating the Key Vault secret and applying — which is a configuration change, and therefore is a replacement — then confirm the fingerprint changed.

6 · The consumer group this resource makes you state

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry" # REQUIRED — no default

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

✅ the_consumer_group_is_required_here_and_optional_on_the_event_hub_inputs is true, and it is the one respect in which this resource is stricter than its siblings. Both event hub inputs make the argument optional, where omitting it silently selects $Default and puts the job into competition with every other reader that did the same.

💡 Nothing here creates the consumer group. Give each reader its own.

7 · The endpoint, and what is actually checked

module "operations_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-ops"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/operationsMonitoringEvents"
  eventhub_consumer_group_name = "sa-ops"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

output "is_telemetry" {
  value = module.operations_input.reads_the_builtin_telemetry_endpoint
  # false — this reads the operations-monitoring feed, not device telemetry
}

⚠️ the_endpoint_value_is_not_validated_by_anything is true. The provider applies only a not-empty check, and this module deliberately does not enforce a closed set: the endpoint vocabulary belongs to the IoT Hub service rather than the Terraform schema, and a list that was wrong would reject a legal value — while a validation {} failure also blocks terraform destroy.

🔒 What the module does reject is the certainly-wrong shape: a leading /, or a full URL. A plausible-but-unrecognised value is accepted and reported.

8 · Csv telemetry

module "csv_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry-csv"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry-csv"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type            = "Csv"
    encoding        = "UTF8"
    field_delimiter = "\t"
  }
}

⚠️ Csv requires both encoding and field_delimiter. The provider errors at apply, inside its expander, after it has already queried Azure for a name collision — so the first error you see may be the wrong one. This module rejects the incomplete pairing at validate instead.

💡 The five legal delimiters are a space, a comma, a tab, a vertical bar and a semicolon. Write the tab as "\t" in HCL.

9 · Avro, and a field that is quietly discarded

module "avro_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry-avro"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry-avro"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  # Both of these are ACCEPTED on an Avro input and then dropped.
  serialization = {
    type            = "Avro"
    encoding        = "UTF8"
    field_delimiter = ","
  }
}

output "dropped" {
  value = module.avro_input.serialization_fields_that_are_accepted_and_dropped
  # ["field_delimiter", "encoding"]
}

💡 The input expander reads only the fields its own branch names, so these two never reach Azure and nothing raises an error. The module reports rather than refuses: refusing would reject a configuration the provider accepts, and a validation {} failure blocks terraform destroy too.

⚠️ The same combination on an output resource is an apply error, because that expander is closed in both directions.

10 · Several IoT Hub inputs on one job

locals {
  hub_feeds = {
    telemetry = { endpoint = "messages/events", group = "sa-telemetry" }
    ops       = { endpoint = "messages/operationsMonitoringEvents", group = "sa-ops" }
  }
}

module "hub_inputs" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"
  for_each = local.hub_feeds

  name                      = "in-${each.key}"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = each.value.endpoint
  eventhub_consumer_group_name = each.value.group

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

💡 for_each over a keyed map rather than count, so removing ops never re-indexes the other. Each input gets its own consumer group, which is the point.

🔒 Both share one policy key, so the composition's blast radius is that policy's — not one feed's.

11 · Timeouts, and which ones a rotation uses

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }

  timeouts = {
    create = "10m"
    read   = "2m"
    update = "10m"
    delete = "5m"
  }
}

✅ all_four_timeouts_are_honoured_here is true. Worth stating because it is not uniform across the family: the blob reference input declares the same four fields and routes every function through the create/update helper, which leaves its read and delete inert.

⚠️ A key rotation exercises delete and create, not update, because the key is force-new. Sizing update for a rotation would size the wrong one.

12 · Importing, and the check that is missing

import {
  to = module.telemetry_input.azurerm_stream_analytics_stream_input_iothub.this
  id = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-streaming/providers/Microsoft.StreamAnalytics/streamingJobs/job-telemetry/inputs/in-telemetry"
}

🔴 the_importer_does_not_check_the_datasource_type is true. This resource's importer validates only that the string parses as an Input Resource ID — it does not read the record. So pointing this at a blob or event hub input succeeds silently, and every subsequent plan then wants to rewrite the datasource, a change that never converges.

⚠️ Several siblings do check and would have refused: the blob reference input, the Service Bus outputs and the event hub V2 input. Confirm what kind of input you are importing before you do it.

💡 The key cannot be imported — Azure never returns it — so it must be supplied in configuration immediately after.

13 · Why a clean plan is not proof the input works

module "misconfigured" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "group-that-does-not-exist"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

🔴 This applies cleanly. nothing_here_creates_the_iot_hub is true: the module does not create the hub, enable its endpoint, create the consumer group, verify any of them exists, or check the credential. A misspelled consumer group, a deleted hub and a regenerated key all apply without error.

⚠️ nothing_verifies_that_the_query_reads_from_this_input is true as well — an input can exist, apply cleanly, and be referenced by no query at all.

14 · 🏗️ End-to-end composition

module "rg" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"

  name     = "rg-streaming"
  location = "eastus"
}

module "hub" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-iothub.git?ref=v1.0.0"

  name                = "iot-telemetry-01"
  resource_group_name = module.rg.name
  location            = module.rg.location

  sku = {
    name     = "S1"
    capacity = 1
  }
}

module "vault" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-key-vault.git?ref=v1.0.0"

  name                = "kv-streaming-01"
  resource_group_name = module.rg.name
  location            = module.rg.location
  tenant_id           = data.azurerm_client_config.current.tenant_id
}

data "azurerm_key_vault_secret" "iot_service_key" {
  name         = "iot-service-policy-key"
  key_vault_id = module.vault.id
}

module "job" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-job.git?ref=v1.0.0"

  name                = "job-telemetry"
  resource_group_name = module.rg.name
  location            = module.rg.location

  transformation_query = <<-QUERY
    SELECT
      deviceId,
      AVG(temperature) AS avgTemperature
    INTO [out-results]
    FROM [in-telemetry]
    GROUP BY deviceId, TumblingWindow(minute, 5)
  QUERY
}

module "telemetry_input" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-stream-analytics-stream-input-iothub.git?ref=v1.0.0"

  name                      = "in-telemetry"
  stream_analytics_job_name = module.job.name
  resource_group_name       = module.rg.name

  iothub_namespace             = module.hub.name
  endpoint                     = "messages/events"
  eventhub_consumer_group_name = "sa-telemetry"

  shared_access_policy_name = "service"
  shared_access_policy_key  = data.azurerm_key_vault_secret.iot_service_key.value

  serialization = {
    type     = "Json"
    encoding = "UTF8"
  }
}

output "streaming_posture" {
  value = {
    input_id        = module.telemetry_input.id
    secret_in_state = module.telemetry_input.a_secret_is_in_state_unconditionally
    no_msi_option   = module.telemetry_input.this_resource_offers_no_managed_identity_option
    over_privileged = module.telemetry_input.uses_the_hub_owner_policy
    grant_scope     = module.telemetry_input.stream_analytics_job_id
  }
}

🔴 The query is the load-bearing part and it is not managed here. FROM [in-telemetry] matches this module's name output by string. Terraform sees no dependency between them.

🔒 This composition cannot be made secret-free. a_secret_is_in_state_unconditionally is true, so the honest controls are an encrypted backend, a narrow policy, a scoped listkeys permission, and a deliberate decision about who holds plan rights.

📥 Inputs

Identity — name, stream_analytics_job_name, resource_group_name (all required, all force-new). Source — iothub_namespace, endpoint, eventhub_consumer_group_name (all required). Credential — shared_access_policy_name, shared_access_policy_key (both required; the key is also force-new and sensitive). Framing — serialization (required). Tail — timeouts.

Full schemas

variable "serialization" {
  type = object({
    type            = string           # Avro | Csv | Json  (no Parquet on any input)
    encoding        = optional(string) # "UTF8" only; REQUIRED for Csv and Json
    field_delimiter = optional(string) # " " | "," | tab | "|" | ";"; REQUIRED for Csv
  })
}

variable "timeouts" {
  type = object({
    create = optional(string) # a key rotation uses this one, plus delete
    read   = optional(string)
    update = optional(string) # NOT used by a key rotation
    delete = optional(string) # a key rotation uses this one, plus create
  })
  default = null
}

🧾 Outputs

Output	Description	Notes
`id`	Resource ID of the input record	first
`name`	The identifier the query reads FROM
`stream_analytics_job_name`	Parent job name
`resource_group_name`	Resource group of the JOB
`stream_analytics_job_id`	Parent job Resource ID	RBAC scope
`iothub_namespace`	The IoT Hub
`endpoint`	The endpoint read from
`reads_the_builtin_telemetry_endpoint`	True for `messages/events`
`eventhub_consumer_group_name`	Consumer group, always explicit
`reads_through_the_default_consumer_group`	True for `$Default`
`shared_access_policy_name`	Policy name	not a secret
`uses_the_hub_owner_policy`	True for `iothubowner`
`shared_access_policy_key_fingerprint`	SHA-256 of the key	never the key
`a_secret_is_in_state_unconditionally`	Constant `true`
`this_resource_offers_no_managed_identity_option`	Constant `true`
`plan_access_is_credential_access`	Constant `true`
`sensitive_redacts_the_plan_and_does_not_encrypt_state`	Constant `true`
`serialization_type`	Avro, Csv or Json
`serialization_encoding`	UTF8 or null
`serialization_field_delimiter`	Delimiter or null
`serialization_fields_that_are_accepted_and_dropped`	Set, then discarded
`rotating_the_key_replaces_the_input`	Constant `true`
`the_force_new_marking_does_not_catch_an_out_of_band_rotation`	Constant `true`
`the_importer_does_not_check_the_datasource_type`	Constant `true`
`nothing_here_creates_the_iot_hub`	Constant `true`
`the_endpoint_value_is_not_validated_by_anything`	Constant `true`
`the_consumer_group_is_required_here_and_optional_on_the_event_hub_inputs`	Constant `true`
`a_key_rotated_outside_terraform_is_invisible`	Constant `true`
`an_iothub_policy_key_grants_what_the_policy_grants`	Constant `true`
`nothing_verifies_that_the_query_reads_from_this_input`	Constant `true`
`the_serialization_type_pairing_is_enforced_in_one_direction_only`	Constant `true`
`create_refuses_an_existing_input`	Constant `true`
`the_import_guard_can_be_disabled_by_a_provider_feature`	Constant `true`
`all_four_timeouts_are_honoured_here`	Constant `true`
`force_new_fields`	Four, including the key
`force_new_fields_inherited_from_shared_schema_helpers`	The one a grep misses
`fields_azure_returns_on_read`	Where drift is visible
`this_resource_supports_no_azure_resource_tags`	Constant `true`

No output carries the shared access policy key.

🧠 Architecture Notes

This is an input that cannot be made secret-free. The provider marks both credential fields Required and exposes no authentication_mode argument, so unlike the event hub inputs — where "Msi" plus an omitted key is a complete configuration — there is no identity route here at all. That distinction matters when reading this suite's other Stream Analytics modules: their key-free defaults are not available here, and a pipeline that must avoid stored keys has to reach Stream Analytics a different way.

The key is force-new, which no other credential in this family is. Changing shared_access_policy_key produces a destroy-and-create plan rather than an update. Combined with the fact that a Stream Analytics job must be stopped before its inputs can change, a rotation is a planned operational event. The policy name is not force-new, so the two halves of one credential pair behave differently — worth knowing before writing a rotation runbook.

Force-new and drift detection are different things, and conflating them produces false confidence. The force-new marking fires on a change Terraform can see in configuration. A key regenerated in the portal is not such a change: Azure never returns the key, the provider has nothing to compare, and the plan is empty. So the marking protects the apply path and does nothing at all for out-of-band rotation. The module emits both facts separately for exactly that reason.

The importer is weaker here than on several siblings. It validates only that the supplied string parses as an Input Resource ID; it does not read the record. Importing a blob or event hub input at this address succeeds, and the resulting configuration then wants to rewrite the datasource on every plan — which never converges, because the two are different datasource types. The blob reference input, the Service Bus outputs and the event hub V2 input all read the record and refuse a mismatch.

One argument is stricter here than on the siblings. eventhub_consumer_group_name is Required, where both event hub inputs make it Optional and silently fall through to $Default. That removes the most common accidental-contention failure in the family: two readers competing for the same partition leases because neither named a group.

The endpoint is documented rather than enforced. The provider checks only that it is non-empty, and this module deliberately does not add a closed set: the endpoint vocabulary belongs to the IoT Hub service, is not published in the Terraform schema, and a wrong list would reject legal input while also blocking terraform destroy. What the module rejects is the certainly-wrong shape — a leading slash or a URL — and it reports whether the value is the expected messages/events.

Force-new is four fields, and a grep of the resource file finds three. resource_group_name inherits its force-new from a shared schema helper, so it never appears in a search of the resource's own definition. The module emits the full set and the inherited subset separately.

🧱 Design Principles

Concern	This module's position	Why
Credential	🔴 The secure-by-default rule cannot apply. Both credential fields are Required and no identity option exists, so there is no empty call to make safe	stated explicitly, validated, and emitted as flags rather than glossed
Secret emission	the key is never emitted; only a SHA-256 fingerprint	re-emitting would copy it into every consuming state
Blast radius	documented: the key grants whatever its policy grants, and `iothubowner` is flagged	nothing in Azure narrows a key below its policy
Access	`plan_access_is_credential_access` is emitted	it changes who should hold plan rights
Enforcement	mirror what the provider enforces; document what it does not (`endpoint`)	never invent a constraint that could reject legal input
Probable mistakes	reject a URL or leading slash on `endpoint`, and an ID where a name belongs	catching the likely error beats catching nothing
Rotation	emit both the force-new fact and its limit	the two are easily conflated into false confidence

🔒 Where a provider makes a credential-bearing argument optional, this suite defaults to the key-free shape — as terraform-azurerm-stream-analytics-stream-input-eventhub does. Where it makes it required and offers no alternative, as here, the rule has nothing to work with and the module says so instead.

🚀 Runbook

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module with ?ref=v1.0.0 — never a branch. This module is plan-only from a workstation; a human applies from CI.

🧪 Testing

terraform validate and terraform fmt -check are the offline proof gate. All 18 validation {} blocks fire at plan, including the two serialization pairing rules the provider itself defers to apply. Each was exercised with a negative fixture that fires it and a fully-populated positive fixture that does not, driven from .tfvars files through terraform console — which, unlike terraform validate on a calling configuration, does evaluate root-module variable validations. Three of the positives exist to prove the module does not over-reach: one passes the operations-monitoring endpoint, one passes $Default as the consumer group, and one passes iothubowner — all legal, all reported rather than refused.

What only a real plan/apply exercises: whether the job exists, whether the IoT Hub, its endpoint and the consumer group exist, whether the policy key is valid, and whether the input name collides.

What nothing exercises, at any stage: whether the transformation query reads from this input's name, and whether the key has been regenerated out of band. Both are why those facts are emitted as outputs rather than assumed.

💬 Example Output

id                                        = "/subscriptions/.../streamingJobs/job-telemetry/inputs/in-telemetry"
name                                      = "in-telemetry"
stream_analytics_job_id                   = "/subscriptions/.../streamingJobs/job-telemetry"
iothub_namespace                          = "iot-telemetry-01"
endpoint                                  = "messages/events"
reads_the_builtin_telemetry_endpoint      = true
eventhub_consumer_group_name              = "sa-telemetry"
shared_access_policy_name                 = "service"
uses_the_hub_owner_policy                 = false
shared_access_policy_key_fingerprint      = "a251c2d14fe333e779dbed2fb61e665dbe1ad1345fda41045f287824d667a3c7"
a_secret_is_in_state_unconditionally      = true
rotating_the_key_replaces_the_input       = true
force_new_fields                          = ["name", "stream_analytics_job_name", "resource_group_name", "shared_access_policy_key"]

🔍 Troubleshooting

Symptom	Cause	Fix
The job reads nothing and nothing errored	The consumer group or endpoint is wrong, or the policy lacks service connect	Nothing here checks any of them; `nothing_here_creates_the_iot_hub` is `true`
A key change produced a destroy-and-create plan	`shared_access_policy_key` is force-new	Expected — `rotating_the_key_replaces_the_input` is `true`; stop the job first
A key was regenerated in the portal and nothing changed in Terraform	Azure never returns the key, so there is nothing to compare	`the_force_new_marking_does_not_catch_an_out_of_band_rotation` is `true`; update the secret and apply
An imported input replans forever	The record is a blob or event hub input; this importer does not check	`the_importer_does_not_check_the_datasource_type` is `true`; import the right kind
A security review asks for managed identity	This resource has no `authentication_mode` argument	`this_resource_offers_no_managed_identity_option` is `true`; the event hub input has one
`serialization.type is "Csv", which requires BOTH...`	Csv without `encoding` or `field_delimiter`	Set both; the provider errors on this at apply
A `field_delimiter` on an Avro input does nothing	It does not apply to the type, so the expander drops it	Check `serialization_fields_that_are_accepted_and_dropped`
`endpoint is a URL or starts with a "/"`	A full URL or an absolute path was passed	Use a relative endpoint — almost always `messages/events`
Two jobs interfere with each other's progress	Both read through the same consumer group	Give each its own; `reads_through_the_default_consumer_group` reports the `$Default` case
`A resource with the ID ... already exists`	An input of this name exists on the job	`terraform import` it; do not rename

🔗 Related Docs

azurerm_stream_analytics_stream_input_iothub
azurerm_stream_analytics_job
Stream data as input into Azure Stream Analytics
Sibling modules: terraform-azurerm-stream-analytics-job, terraform-azurerm-stream-analytics-stream-input-eventhub, terraform-azurerm-stream-analytics-output-eventhub, terraform-azurerm-iothub, terraform-azurerm-key-vault
This module's SCOPE.md

💙 "Infrastructure as Code should be standardized, consistent, and secure."

Navigation Menu

Repository files navigation

☁️ Azure Stream Analytics IoT Hub Stream Input Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Latest commit

History

Folders and files

Repository files navigation

☁️ Azure Stream Analytics IoT Hub Stream Input Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages