☁️ Azure Kusto Event Grid Data Connection Terraform Module

A standalone Terraform module that provisions a single azurerm_kusto_eventgrid_data_connection, ingesting Storage blobs into a Kusto (Azure Data Explorer) table via Event Grid notifications routed through an Event Hub. Targets hashicorp/azurerm ~> 4.0.

🧩 Overview

This module manages one Kusto Event Grid data connection and nothing else:

📥 Wires Storage blob events into a Kusto database table for continuous ingestion.
🔀 Routes storage notifications through an Event Hub consumer group so bursts are buffered reliably.
🧾 Selects the blob event type (BlobCreated vs BlobRenamed), the source data format, and the mapping/table applied at ingest time.
🪪 Prefers a managed identity for the ingestion path instead of shared credentials.
🧭 Supports single- or multi-database routing, validated at parse time.

💡 Why it matters: A Kusto cluster is only as useful as the data flowing into it. This module gives you a typed, secure-by-default ingestion edge that binds an existing Storage account, Event Hub, and Kusto database together, so a composition can stand up an event-driven analytics pipeline without hand-wiring the connection.

❤️ Support this project

If this module saves you time, please consider supporting it:

⭐ Star the repo on GitHub — it helps others find the library.
💼 Connect on LinkedIn — linkedin.com/in/microsoftexpert
☕ Buy me a coffee — buymeacoffee.com/microsoftexpert

🗺️ Where this fits in the family

flowchart LR
  cluster["terraform-azurerm-kusto-cluster"]
  db["terraform-azurerm-kusto-database"]
  adc["terraform-azurerm-kusto-attached-database-configuration"]
  script["terraform-azurerm-kusto-script"]
  cosmos["terraform-azurerm-kusto-cosmosdb-data-connection"]
  eg["terraform-azurerm-kusto-eventgrid-data-connection"]
  eh["terraform-azurerm-kusto-eventhub-data-connection"]
  iot["terraform-azurerm-kusto-iothub-data-connection"]
  cmk["terraform-azurerm-kusto-cluster-customer-managed-key"]
  mpe["terraform-azurerm-kusto-cluster-managed-private-endpoint"]
  cpa["terraform-azurerm-kusto-cluster-principal-assignment"]
  dpa["terraform-azurerm-kusto-database-principal-assignment"]
  cluster -->|"hosts"| db
  cluster -->|"attaches leader db"| adc
  db -->|"target of"| script
  db -->|"ingests via"| cosmos
  db -->|"ingests via"| eg
  db -->|"ingests via"| eh
  db -->|"ingests via"| iot
  cluster -->|"encrypted by, BY ID"| cmk
  cluster -->|"private egress via, BY NAME"| mpe
  cluster -->|"data-plane roles: ALL databases"| cpa
  db -->|"data-plane roles: ONE database, prefer this"| dpa
  mpe -->|"makes the source reachable"| eh
  classDef me fill:#0078D4,stroke:#004578,color:#ffffff;
  classDef ext fill:#f2f2f2,stroke:#c8c8c8,color:#111111;
  class eg me;
  class cluster,db,adc,script,cosmos,eh,iot,cmk,mpe,cpa,dpa ext;

This module is one of the Kusto data-connection siblings (Cosmos DB / Event Grid / Event Hub / IoT Hub). It targets an existing azurerm_kusto_database and consumes an existing Storage account and Event Hub by id.

🧬 What this module builds

flowchart LR
  in_id["name / location / resource_group_name / cluster_name / database_name"]
  in_src["storage_account_id / eventhub_id / eventhub_consumer_group_name"]
  in_opt["blob_storage_event_type / data_format / managed_identity_id"]
  res["azurerm_kusto_eventgrid_data_connection.this"]
  out_id["id"]
  out_name["name"]
  in_id -->|"input"| res
  in_src -->|"input"| res
  in_opt -->|"input"| res
  res -->|"output"| out_id
  res -->|"output"| out_name
  classDef me fill:#0078D4,stroke:#004578,color:#ffffff;
  class res me;

Resource inventory

Resource	Cardinality	Role
`azurerm_kusto_eventgrid_data_connection.this`	single (keystone)	The Event Grid data connection bound to a Kusto database.

The module owns a single resource. Its outputs go well beyond id and name, because several facts about this connection are invisible in state -- see the Outputs table.

✅ Provider / Versions

Requirement	Value
Terraform	`>= 1.12.0`
Provider	`hashicorp/azurerm ~> 4.0`
Provider block	None in this module — the caller configures `provider "azurerm" { features {} }`, auth, subscription, and region.

Schema notes that bite:

🔒 Force-new fields: name, location, resource_group_name, cluster_name, database_name, storage_account_id, eventhub_id, eventhub_consumer_group_name, and database_routing_type all force replacement when changed. Treat the connection's identity and its source bindings as immutable.
🎯 blob_storage_event_type accepts only Microsoft.Storage.BlobCreated or Microsoft.Storage.BlobRenamed; anything else is rejected at parse time.
🧱 mapping_rule_name and table_name, when set, should already exist in the target database before the connection is created -- the connection references them and does not create them. Nothing verifies that the table or mapping exists -- not validate, not plan, not apply. Both fields carry only a FORMAT validator, and the provider never calls the Kusto validation API, so ARM stores whatever string you give it and returns success. A wrong name produces a connection that looks correct everywhere and ingests nothing.
🧾 data_format is validated against the Kusto-supported format set (APACHEAVRO, AVRO, CSV, JSON, MULTIJSON, ORC, PARQUET, PSV, RAW, SCSV, SINGLEJSON, SOHSV, TSV, TSVE, TXT, W3CLOGFILE).
🪪 managed_identity_id takes the cluster resource ID for a system-assigned identity, or the identity's resource ID for a user-assigned identity.

🔑 Required Azure RBAC Roles / Permissions

Least-privilege at the Kusto cluster/database scope:

Microsoft.Kusto/clusters/databases/dataConnections/write
Microsoft.Kusto/clusters/databases/dataConnections/read
Microsoft.Kusto/clusters/databases/dataConnections/delete

The connection's identity additionally needs read on the Storage account and Event Hub. The built-in Contributor role on the cluster covers the connection operations.

🧰 Azure Prerequisites

An existing Kusto cluster and database, a Storage account, and an Event Hub with a consumer group.
An Event Grid subscription wiring the storage events to the Event Hub (supply its ID via eventgrid_event_subscription_id, or let Azure create one).
The Microsoft.Kusto resource provider registered on the subscription.
If mapping_rule_name or table_name is set, the mapping and table must already exist in the target database.

📁 Module Structure

terraform-azurerm-kusto-eventgrid-data-connection/
├── providers.tf     # required_version + azurerm ~> 4.0 pin; no provider block
├── variables.tf     # typed inputs; enum validations; timeouts tail (no tags)
├── main.tf          # keystone azurerm_kusto_eventgrid_data_connection.this
├── outputs.tf       # id (first), then name
├── README.md        # this document
├── SCOPE.md         # cross-module contract
├── LICENSE          # MIT
└── .gitignore       # canonical library ignore set

⚙️ Quick Start

provider "azurerm" {
  features {}
}

module "kusto_eventgrid_ingest" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "blob-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-analytics/providers/Microsoft.Storage/storageAccounts/adxlanding"
  eventhub_id                  = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-analytics/providers/Microsoft.EventHub/namespaces/adx-ns/eventhubs/blob-events"
  eventhub_consumer_group_name = "adx-ingest"
}

ℹ️ The caller owns the provider, its authentication, subscription/region selection, and the mandatory features {} block. This module never declares a provider "azurerm" block.

🔌 Cross-Module Contract

Consumes

Input	Type	Source module
`resource_group_name`	`string`	`terraform-azurerm-resource-group`
`cluster_name` / `database_name`	`string`	`terraform-azurerm-kusto-cluster` / `terraform-azurerm-kusto-database`
`storage_account_id`	`string`	`terraform-azurerm-storage-account`
`eventhub_id`	`string`	`terraform-azurerm-eventhub-namespace` (an event hub)
`managed_identity_id`	`string`	`terraform-azurerm-user-assigned-identity` (optional)

Emits

Output	Description	Consumed by
`id`	Resource ID of the data connection	references
`name`	Connection name	references

📚 Example Library

1 · Minimal blob-created ingestion

The smallest real call — every required binding, provider defaults for everything optional.

module "ingest_min" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "blob-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
}

💡 With no blob_storage_event_type set, the connection uses the provider default (Microsoft.Storage.BlobCreated).

2 · Explicit BlobCreated events

Pin the event type to newly created blobs for a write-once landing zone.

module "ingest_created" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "created-ingest"
  location                     = "eastus2"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  blob_storage_event_type      = "Microsoft.Storage.BlobCreated"
}

ℹ️ BlobCreated fires on new-blob writes — the common pattern for append-only ingestion.

3 · BlobRenamed events (ADLS Gen2 rename)

Ingest on the rename event, useful when a writer stages a file and atomically renames it into place on a hierarchical-namespace account.

module "ingest_renamed" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "renamed-ingest"
  location                     = "westus2"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  blob_storage_event_type      = "Microsoft.Storage.BlobRenamed"
}

⚠️ BlobRenamed requires an ADLS Gen2 (hierarchical namespace) account; block-blob-only accounts do not emit it.

4 · JSON data format with a mapping rule

Bind a pre-created JSON mapping and target table.

module "ingest_json" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "json-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  data_format                  = "JSON"
  mapping_rule_name            = "events_json_mapping"
  table_name                   = "RawEvents"
}

🔒 mapping_rule_name and table_name must already exist in the database — create them with a Kusto script before this connection.

5 · MULTIJSON for line-delimited payloads

Use MULTIJSON when each blob holds multiple JSON objects.

module "ingest_multijson" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "multijson-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  data_format                  = "MULTIJSON"
  table_name                   = "RawEvents"
}

💡 MULTIJSON parses a stream of JSON objects; JSON expects one object per line.

6 · CSV with skip_first_record for headers

Skip the header row of every CSV file.

module "ingest_csv" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "csv-ingest"
  location                     = "centralus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  data_format                  = "CSV"
  skip_first_record            = true
  table_name                   = "SalesCsv"
}

ℹ️ skip_first_record left null uses the provider default (false).

7 · PARQUET columnar ingestion

Point the connection at Parquet blobs.

module "ingest_parquet" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "parquet-ingest"
  location                     = "eastus2"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  data_format                  = "PARQUET"
  mapping_rule_name            = "events_parquet_mapping"
  table_name                   = "RawEvents"
}

💡 Columnar formats (PARQUET, ORC, AVRO) usually pair with a mapping rule that names the source columns.

8 · System-assigned managed identity

Use the cluster's system-assigned identity for the ingestion path — supply the cluster resource ID.

module "ingest_sysid" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "sysid-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  managed_identity_id          = azurerm_kusto_cluster.adx.id
}

🔒 Prefer managed identity over shared keys — the cluster identity needs read on the Storage account and Event Hub.

9 · User-assigned managed identity

Use a dedicated user-assigned identity — supply the identity's resource ID.

module "ingest_uami" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "uami-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  managed_identity_id          = azurerm_user_assigned_identity.ingest.id
}

ℹ️ The user-assigned identity must be attached to the cluster and granted read on the source resources.

10 · Multi-database routing

Allow the connection to route to a database named in the ingestion payload rather than only the bound database.

module "ingest_multi" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "multi-route-ingest"
  location                     = "eastus"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"
  database_routing_type        = "Multi"
}

⚠️ database_routing_type is force-new — flipping between Single and Multi replaces the connection.

11 · Explicit Event Grid subscription ID

Reference an Event Grid subscription you manage separately instead of letting Azure create one.

module "ingest_egsub" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                            = "egsub-ingest"
  location                        = "eastus"
  resource_group_name             = "rg-analytics"
  cluster_name                    = "adx-prod"
  database_name                   = "telemetry"
  storage_account_id              = azurerm_storage_account.landing.id
  eventhub_id                     = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name    = "adx-ingest"
  eventgrid_event_subscription_id = azurerm_eventgrid_event_subscription.blobs.id
}

💡 Supplying the subscription ID keeps the Event Grid wiring in your own state rather than as a side effect.

12 · Custom timeouts

Extend the create/delete windows for a slow control-plane region.

module "ingest_timeouts" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "timeout-ingest"
  location                     = "westus2"
  resource_group_name          = "rg-analytics"
  cluster_name                 = "adx-prod"
  database_name                = "telemetry"
  storage_account_id           = azurerm_storage_account.landing.id
  eventhub_id                  = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name = "adx-ingest"

  timeouts = {
    create = "60m"
    delete = "60m"
  }
}

ℹ️ Timeouts are Go duration strings; unset operations fall back to the provider defaults.

13 · Fully specified connection

Every knob set — event type, format, mapping, table, routing, identity, and skip.

module "ingest_full" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                            = "full-ingest"
  location                        = "eastus"
  resource_group_name             = "rg-analytics"
  cluster_name                    = "adx-prod"
  database_name                   = "telemetry"
  storage_account_id              = azurerm_storage_account.landing.id
  eventhub_id                     = azurerm_eventhub.blob_events.id
  eventhub_consumer_group_name    = "adx-ingest"
  blob_storage_event_type         = "Microsoft.Storage.BlobCreated"
  data_format                     = "JSON"
  database_routing_type           = "Single"
  eventgrid_event_subscription_id = azurerm_eventgrid_event_subscription.blobs.id
  managed_identity_id             = azurerm_user_assigned_identity.ingest.id
  mapping_rule_name               = "events_json_mapping"
  table_name                      = "RawEvents"
  skip_first_record               = false
}

🔒 A fully declarative connection keeps the ingestion contract in code and out of the Kusto portal.

14 · 🏗️ End-to-end composition

Wire sibling module outputs into this module — resource group, storage account, Event Hub namespace, user-assigned identity, Kusto cluster, and Kusto database all feed the connection.

provider "azurerm" {
  features {}
}

module "rg" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"
  name     = "rg-analytics"
  location = "eastus"
}

module "identity" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-user-assigned-identity.git?ref=v1.0.0"
  name                = "id-adx-ingest"
  location            = module.rg.location
  resource_group_name = module.rg.name
}

module "storage" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-storage-account.git?ref=v1.0.0"
  name                = "adxlanding001"
  location            = module.rg.location
  resource_group_name = module.rg.name
}

module "eventhub" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-eventhub-namespace.git?ref=v1.0.0"
  name                = "adx-ns"
  location            = module.rg.location
  resource_group_name = module.rg.name

  # Quote any key containing a hyphen: HCL reads a bare blob-events as the
  # expression `blob - events`, which is a parse error, not a plan error.
  event_hubs = {
    "blob-events" = {
      name              = "blob-events"
      partition_count   = 4
      message_retention = 1
    }
  }

  # consumer_groups is a SEPARATE top-level variable, not an attribute of an
  # event_hubs entry -- nested there it is silently discarded and no consumer
  # group is created, while the connection below still names one.
  consumer_groups = {
    "adx-ingest" = {
      name         = "adx-ingest"
      eventhub_key = "blob-events"
    }
  }
}

module "kusto_cluster" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-cluster.git?ref=v1.0.0"

  sku                   = {
    name     = "Standard_E8ads_v5"
    capacity = 2
  }
  name                = "adxprod001"
  location            = module.rg.location
  resource_group_name = module.rg.name
}

module "kusto_db" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-database.git?ref=v1.0.0"
  name                = "telemetry"
  location            = module.rg.location
  resource_group_name = module.rg.name
  cluster_name        = module.kusto_cluster.name
}

module "eventgrid_ingest" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-kusto-eventgrid-data-connection.git?ref=v1.0.0"

  name                         = "blob-ingest"
  location                     = module.rg.location
  resource_group_name          = module.rg.name
  cluster_name                 = module.kusto_cluster.name
  database_name                = module.kusto_db.name
  storage_account_id           = module.storage.id
  eventhub_id                  = module.eventhub.eventhub_ids["blob-events"]
  eventhub_consumer_group_name = "adx-ingest"
  blob_storage_event_type      = "Microsoft.Storage.BlobCreated"
  data_format                  = "JSON"
  managed_identity_id          = module.identity.id
  mapping_rule_name            = "events_json_mapping"
  table_name                   = "RawEvents"
  skip_first_record            = false
}

ℹ️ Output names shown for sibling modules (eventhub_ids, id, name, location) follow this suite's conventions; align them with the exact outputs of the versions you pin. The mapping rule and table should be created in the database (for example via a Kusto script) before this connection -- nothing enforces the ordering.

📥 Inputs

Required — the connection identity and its source bindings.

Name	Type	Description
`name`	`string`	Connection name. Force-new.
`location`	`string`	Azure region of the Kusto database. Force-new.
`resource_group_name`	`string`	Resource group of the cluster/database. Force-new.
`cluster_name`	`string`	Kusto cluster name. Force-new.
`database_name`	`string`	Kusto database name. Force-new.
`storage_account_id`	`string`	Storage account resource ID whose blob events drive ingestion. Force-new.
`eventhub_id`	`string`	Event Hub resource ID carrying the storage events. Force-new.
`eventhub_consumer_group_name`	`string`	Event Hub consumer group used for ingestion. Force-new.

Optional — ingestion shaping, identity, and the timeouts tail.

Name	Type	Default	Description
`blob_storage_event_type`	`string`	`null`	`Microsoft.Storage.BlobCreated` or `Microsoft.Storage.BlobRenamed`.
`data_format`	`string`	`null`	Source data format (validated enum).
`database_routing_type`	`string`	`null`	`Single` or `Multi`. Force-new.
`eventgrid_event_subscription_id`	`string`	`null`	Event Grid subscription resource ID.
`managed_identity_id`	`string`	`null`	Managed identity resource ID for ingestion.
`mapping_rule_name`	`string`	`null`	Existing mapping rule applied at ingest.
`table_name`	`string`	`null`	Existing target table.
`skip_first_record`	`bool`	`null`	Ignore the first record of every file.
`timeouts`	`object(...)`	`null`	Per-operation timeouts.

Full input schemas

variable "blob_storage_event_type" {
  type    = string
  default = null
  # "Microsoft.Storage.BlobCreated" | "Microsoft.Storage.BlobRenamed"
}

variable "data_format" {
  type    = string
  default = null
  # APACHEAVRO | AVRO | CSV | JSON | MULTIJSON | ORC | PARQUET | PSV |
  # RAW | SCSV | SINGLEJSON | SOHSV | TSV | TSVE | TXT | W3CLOGFILE
}

variable "database_routing_type" {
  type    = string
  default = null
  # "Single" | "Multi" — force-new
}

variable "skip_first_record" {
  type    = bool
  default = null
}

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

ℹ️ This resource does not support tags, so the universal tail is timeouts only.

🧾 Outputs

Output	Description	Kind
`id`		Passthrough
`name`	Name of the data connection, unique within the target Kusto database	Passthrough
`cluster_name`	Name of the Kusto cluster hosting this connection	Passthrough
`database_name`	Name of the Kusto database this connection ingests into by default	Passthrough
`resource_group_name`	Resource group containing the parent Kusto cluster	Passthrough
`location`	Azure region of the connection, normalised by the provider ("East US" is stored as "eastus")	Passthrough
`storage_account_id`	Resource ID of the Storage account whose blob events drive ingestion and from which Kusto fetches blob content	Passthrough
`eventhub_id`	Resource ID of the Event Hub carrying the Event Grid notifications	Passthrough
`eventhub_consumer_group_name`	Event Hub consumer group this connection reads from	Passthrough
`eventgrid_event_subscription_id`		Passthrough
`blob_storage_event_type`	"Microsoft.Storage.BlobCreated" or "Microsoft.Storage.BlobRenamed"	Passthrough
`data_format`	Static data format applied to blobs that carry no format metadata of their own	Passthrough
`database_routing_type`		Passthrough
`skip_first_record`	Whether the first record of every blob is discarded	Passthrough
`table_name`	Static target table for blobs carrying no table metadata	Passthrough
`mapping_rule_name`	Static ingestion mapping for blobs carrying no mapping metadata	Passthrough
`managed_identity_id`		Passthrough
`uses_managed_identity`	Whether an explicit managed identity was supplied for the ingestion path	Passthrough
`target_table_and_mapping_are_invisible_to_terraform`		Constant
`in_place_update_applies_every_non_force_new_field`		Constant
`managed_identity_cannot_be_cleared_in_place`		Constant
`malformed_records_can_be_dropped_without_error`		Constant
`read_uses_the_delete_timeout`		Constant
`plan_does_not_require_credential_access`		Constant
`destroy_stops_ingestion_and_deletes_no_data`		Constant
`provider_default_timeouts`	The provider's built-in timeouts for this resource, for reference when deciding whether to override them: create 60m, update 60m, delete 60m, read 5m	Derived

🧠 Architecture Notes

Immutable core. The connection's identity (name), its location and resource group, the cluster and database it targets, the storage/event-hub bindings, and database_routing_type are all force-new. Any change to these replaces the connection, which briefly interrupts ingestion — plan such edits as create-before-destroy where your workflow allows.
The Event Grid → Event Hub → Kusto flow. Blob write/rename events are published to Event Grid, delivered to the Event Hub, and read by Kusto through the named consumer group. This module owns only the Kusto-side connection; the storage account, Event Hub, and (optionally) the Event Grid subscription are referenced by id and created by sibling modules.
Enum validations fail fast. blob_storage_event_type, data_format, and database_routing_type each carry a validation {} block, so a malformed value is a plan-time type error rather than an Azure API rejection mid-apply. A null value defers to the provider default in every case.
Mapping and table are references, not resources. When mapping_rule_name or table_name is set, those objects must already exist in the database; the connection binds to them.
features {} dependence. Like every module in this library, this one declares no provider block. It will not initialize in isolation without a caller-supplied provider "azurerm" { features {} } — that is expected, not a module defect.

🧱 Design Principles

Principle	How this module applies it
Enums validated at parse time	`blob_storage_event_type`, `data_format`, and `database_routing_type` are checked with `validation {}` blocks; illegal values never reach Azure.
Identity over secrets	`managed_identity_id` is the intended ingestion credential path — a system- or user-assigned managed identity, not a shared key.
Least-privilege scope	The RBAC section names only the `dataConnections` verbs at the cluster/database scope.
No tags	The resource does not support `tags`; the universal tail is `timeouts` only, and no `tags` variable is offered.
Safe by omission	Every optional field defaults to `null`, deferring to the provider's own (safe) default rather than inventing one.

🚀 Runbook

# Offline validation — no backend, no cloud calls
terraform init -backend=false
terraform validate
terraform fmt -check

# Plan against a real subscription (a human applies from CI)
terraform plan

Pin the module with ?ref=v1.0.0 — never a branch.
This library is plan-only; terraform apply runs from CI under human review, never from this authoring flow.
Clean up .terraform/ after offline validation.

🧪 Testing

Gate	Covers	Does not cover
`terraform plan`	Type correctness of every input, enum `validation {}` blocks, reference integrity — offline, without credentials. (`terraform validate` alone reaches the types but not the `validation {}` blocks, which it never evaluates through a module call.)	Whether the cluster, database, storage account, or Event Hub actually exist.
`terraform fmt -check`	Canonical formatting.	Semantics.
`terraform plan`	The full resource graph against the live provider, and force-new detection. Not mapping or table existence -- the API surfaces nothing of the kind.	Runtime ingestion -- only an `apply` plus real blob events proves the data path.

💬 Example Output

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

id   = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-analytics/providers/Microsoft.Kusto/clusters/adx-prod/databases/telemetry/dataConnections/blob-ingest"
name = "blob-ingest"

🔍 Troubleshooting

Symptom	Cause	Fix
`blob_storage_event_type must be ...` at plan	A value outside the two allowed event types.	Use `Microsoft.Storage.BlobCreated` or `Microsoft.Storage.BlobRenamed`, or leave it `null`.
`data_format must be one of ...` at plan	An unsupported format string.	Use one of the validated Kusto formats (e.g. `JSON`, `CSV`, `PARQUET`).
Ingestion is silent and no rows land	`mapping_rule_name` / `table_name` name an object that does not exist. The apply did NOT fail -- nothing validates them, so ARM accepted the strings and returned success.	Create the table and mapping in the database first (for example via a Kusto script). Unlike the Event Hub sibling, this resource's update path does apply a corrected name in place.
Plan shows the connection being replaced	A force-new field changed (name, location, RG, cluster, database, storage/event-hub binding, or routing type).	Confirm the change is intended; sequence a create-before-destroy if ingestion continuity matters.
`provider ... features` initialization error	The caller's root module is missing `provider "azurerm" { features {} }`.	Add the `features {}` block to the caller's provider configuration.
No data arrives despite a healthy connection	Event Grid subscription or consumer group not wired to this Event Hub, or the identity lacks read on the source.	Verify the Event Grid subscription targets the Event Hub and grant the ingestion identity read on the storage account and Event Hub.
Access denied during ingestion	The managed identity lacks read on the Storage account or Event Hub.	Grant the identity in `managed_identity_id` read at the source scopes.

🔗 Related Docs

Terraform Registry — azurerm_kusto_eventgrid_data_connection
Terraform Registry — azurerm_kusto_cluster · azurerm_kusto_database
Sibling modules — terraform-azurerm-kusto-cluster, terraform-azurerm-kusto-database, terraform-azurerm-kusto-eventhub-data-connection, terraform-azurerm-kusto-cosmosdb-data-connection, terraform-azurerm-kusto-iothub-data-connection, terraform-azurerm-kusto-script, terraform-azurerm-kusto-attached-database-configuration
This module's cross-module contract — SCOPE.md

💙 "Infrastructure as Code should be standardized, consistent, and secure."

Navigation Menu

Repository files navigation

☁️ Azure Kusto Event Grid Data Connection Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

🧰 Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Latest commit

History

Folders and files

Repository files navigation

☁️ Azure Kusto Event Grid Data Connection Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

🧰 Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages