Skip to content

Latest commit

Β 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ“Š Microsoft Fabric KQL Queryset Terraform Module

Manages a Fabric KQL Queryset item (fabric_kql_queryset) β€” a saved collection of KQL queries typically pointed at one or more KQL databases/Eventhouses for interactive exploration β€” targeting the microsoft/fabric provider ~> 1.12.0.

Terraform Provider Module Version Module Type Resources Posture


🧩 Overview

  • Creates and manages exactly one Fabric KQL Queryset (fabric_kql_queryset.this).
  • Optionally carries a single Default-format definition part (RealTimeQueryset.json) β€” the saved-query JSON payload β€” modeled as a path-keyed Attributes Map, per this module suite's definition-carrying item pattern.
  • Defaults to workspace-root placement (folder_id = null) and no predefined content (definition = {}).
  • Optionally assigns existing fabric_tag GUIDs via tags β€” confirmed present on this resource's live schema.
  • Emits the Queryset id for cross-referencing by sharing automation or downstream orchestration outside Terraform's own reference graph.

πŸ’‘ Why it matters: a KQL Queryset is a low-blast-radius item (saved queries, not data), but its definition payload can still embed a specific database/Eventhouse binding and, via parameters, inject values at plan time β€” modeling it as a strictly-typed Attributes Map catches a malformed part or an illegal processing_mode at terraform validate, not as a 400 from the Fabric API mid-apply.


❀️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


πŸ—ΊοΈ Where this fits

flowchart TB
 Workspace["terraform-fabric-workspace"]:::keystone
 Folder["terraform-fabric-folder"]:::sibling
 Tag["terraform-fabric-tag"]:::sibling
 KqlQueryset["terraform-fabric-kql-queryset\n(this module)"]:::thisModule
 Downstream["Sharing / Real-Time Hub automation\n(outside Terraform)"]:::sibling

 Workspace -->|"workspace_id"| KqlQueryset
 Folder -->|"folder_id (optional)"| KqlQueryset
 Tag -->|"tags (optional set of GUIDs)"| KqlQueryset
 KqlQueryset -->|"id"| Downstream

 classDef thisModule fill:#0F6CBD,color:#FFFFFF,stroke:#0F6CBD
 classDef keystone fill:#143551,color:#FFFFFF,stroke:#143551
 classDef sibling fill:#E8EAED,color:#1A1A1A,stroke:#B0B7BF
Loading

This module never creates the workspace, folder, or tags it references β€” those are sibling modules whose id outputs feed this module's workspace_id / folder_id / tags inputs. The live v1.12.0 schema does not expose a separate top-level reference to a target KQL database/Eventhouse; any such binding is expressed entirely inside the definition JSON payload (see 🧠 Architecture Notes) β€” this is a confirmed schema fact, not an oversight, and there is deliberately no terraform-fabric-kql-database edge in the diagram above.


🧬 What this builds

flowchart TB
 subgraph Inputs["Inputs"]
 DisplayName["display_name"]
 Description["description"]
 WorkspaceId["workspace_id"]
 FolderId["folder_id"]
 Format["format"]
 Definition["definition (Attributes Map)"]
 DefUpdate["definition_update_enabled"]
 Tags["tags"]
 Timeouts["timeouts"]
 end

 Keystone["fabric_kql_queryset.this"]:::thisModule

 DisplayName --> Keystone
 Description --> Keystone
 WorkspaceId --> Keystone
 FolderId --> Keystone
 Format --> Keystone
 Definition -->|"assigned with ="| Keystone
 DefUpdate --> Keystone
 Tags --> Keystone
 Timeouts -->|"assigned with ="| Keystone

 Keystone -->|"id"| OutId["id output"]
 Keystone -->|"display_name"| OutName["display_name output"]

 classDef thisModule fill:#0F6CBD,color:#FFFFFF,stroke:#0F6CBD
Loading

Resource inventory: 1 resource β€” fabric_kql_queryset.this (single instance, the sole keystone; this is a standalone module with no owned children).


βœ… Provider / Versions

Requirement Value
Terraform >= 1.12.0
Provider microsoft/fabric ~> 1.12.0
Provider configuration None in this module β€” the caller configures provider "fabric" {} (auth, tenant) at the root
Preview gate Not required β€” fabric_kql_queryset is a GA resource on the live v1.12.0 schema (no preview banner in the provider docs)
Delegated-auth exception Not applicable β€” the live schema states "This resource supports Service Principal authentication"

Schema notes that bite (confirmed against the live v1.12.0 schema via the terraform-registry MCP, the live provider schema, provider_doc_id 12766478):

  • definition is a Terraform Plugin Framework Attributes Map, not a Block β€” assigned directly with = in main.tf (definition = { for path, part in var.definition: path => {...} }), never wrapped in a dynamic "definition" block. timeouts is likewise a single nested Attributes object, also assigned with =. Both facts are confirmed directly from the tfplugindocs-generated schema text, which labels each (Attributes Map) / (Attributes) respectively β€” a dynamic block on either fails terraform validate against this resource.
  • The schema currently accepts exactly one definition-part key, "RealTimeQueryset.json", paired with format = "Default" β€” the only documented format value. This is a de facto single-entry map today, but the module still models definition as a generic map(object(...)) (not a single nested object) to mirror the provider's own Attributes Map shape and to avoid hard-coding an assumption about future format/part additions.
  • processing_mode's provider-native default is "GoTemplate"; this module overrides its own default to "None" per this module suite's secure-by-default convention, so no implicit token substitution happens unless the caller explicitly opts in.
  • No configuration argument exists on this resource (unlike fabric_lakehouse/fabric_sql_database) β€” do not add one; there is nothing analogous to lakehouse schema-enablement or SQL Database creation mode here.
  • tags is a genuine Set of String on the live schema (tag GUIDs) β€” confirmed present, wired straight through.

πŸ”‘ Required Fabric / Entra Permissions

(sourced from this module's SCOPE.md β€” do not let this drift independently)

  • Fabric workspace role: Admin, Member, or Contributor β€” Microsoft Learn's "Roles in workspaces in Microsoft Fabric" table explicitly lists KQL Querysets under "Write or delete... KQL Querysets...": granted to Admin/Member/Contributor, not Viewer. This matches the Discovery-phase baseline (Workspace Contributor) and is confirmed, not inferred.
  • Entra: the calling Service Principal (or Managed Identity) must be included in the tenant's allowed security group under the "Service principals can call Fabric public APIs" Developer setting β€” the same universal prerequisite every other module in this library carries.
  • No KQL-Queryset-specific Entra app-role or additional API permission beyond the standard Fabric workspace role above was found in the provider docs or Microsoft Learn.

Microsoft Fabric Prerequisites

  • The target capacity backing the workspace must be in Active state (not paused) for KQL Queryset create/read operations β€” Fabric write operations against a paused capacity fail or return stale state.
  • The tenant's "Service principals can call Fabric public APIs" Developer setting must already be enabled in the Fabric Admin Portal before any SPN/MSI auth call succeeds at all, independent of this module's correctness.
  • A KQL Queryset is generally created against an existing KQL database/Eventhouse with editing permissions and data (Microsoft Learn, "Create a KQL queryset" prerequisites) β€” that database is out of this module's scope; the binding is expressed inside definition's JSON payload, not a Terraform argument.
  • No Fabric trial/license prerequisite beyond an active (non-trial, per the provider's own "Known limitations" β€” Fabric trial capacity is unsupported by this provider) capacity being available.

πŸ“ Module Structure

terraform-fabric-kql-queryset/
β”œβ”€β”€ providers.tf # required_providers (fabric ~> 1.12.0), no provider {} block
β”œβ”€β”€ variables.tf # display_name, workspace_id, description, folder_id, format, definition,
β”‚ # definition_update_enabled, tags, timeouts
β”œβ”€β”€ main.tf # fabric_kql_queryset.this β€” the sole keystone
β”œβ”€β”€ outputs.tf # id, display_name
β”œβ”€β”€ README.md # this file
β”œβ”€β”€ SCOPE.md # lightweight cross-module contract (standalone)
└── examples/
 β”œβ”€β”€ basic/ # empty Queryset, no definition, workspace-root placement
 └── complete/ # Queryset with a token-substituted definition, folder_id, tags

βš™οΈ Quick Start

# Caller's root module configures the provider β€” never this module.
provider "fabric" {
  tenant_id     = var.tenant_id
  client_id     = var.client_id
  client_secret = var.client_secret # sourced from Key Vault / pipeline secret, never literal
}

module "kql_queryset" {
  source = "git::https://github.com/microsoftexpert/terraform-fabric-kql-queryset.git?ref=v1.0.0"

  display_name = "kqs-telemetry-ops-prod"
  description  = "Curated telemetry queryset β€” prod. Owner: Observability."
  workspace_id = module.workspace.id
}

πŸ”Œ Cross-Module Contract

Consumes

Input Type Source module
workspace_id string (required) terraform-fabric-workspace id output
folder_id string (optional) terraform-fabric-folder id output
tags set(string) (optional) terraform-fabric-tag id outputs

Emits

Output Description Consumed by
id KQL Queryset GUID Sharing automation, downstream orchestration outside Terraform's reference graph
display_name KQL Queryset display name Informational / cross-referencing

πŸ“š Example Library

1 Β· Safe empty call
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-sandbox-explore-dev"
  workspace_id = module.workspace.id
}
2 Β· Governance context via description

ℹ️ Naming and description remain the primary governance surfaces even though this resource also supports tags β€” see this module suite's naming convention.

module "kql_queryset" {
  source = "../../"

  display_name = "kqs-claims-triage-prod"
  description  = "Triage queries against claims Eventhouse. Owner: Claims Analytics."
  workspace_id = module.workspace.id
}
3 Β· Explicit folder placement
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  folder_id    = module.folder_analysis.id
}
4 Β· Tagged with existing fabric_tag GUIDs
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  tags         = [fabric_tag.this["observability"].id]
}
5 Β· Bootstrap-only definition (no drift tracking)

ℹ️ definition_update_enabled = false bootstraps the Queryset once from source but never re-applies it on later source-content changes β€” useful when the queryset is expected to be hand-edited afterward.

module "kql_queryset" {
  source = "../../"

  display_name              = "kqs-telemetry-ops-dev"
  workspace_id              = module.workspace.id
  definition_update_enabled = false
  format                    = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source = "${path.module}/files/RealTimeQueryset.json"
    }
  }
}
6 Β· Definition with drift tracking (default behavior)
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  format       = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source = "${path.module}/files/RealTimeQueryset.json"
    }
  }
}
7 Β· Token substitution with a custom delimiter
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  format       = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source           = "${path.module}/files/RealTimeQueryset.json.tmpl"
      tokens_delimiter = "<<>>"
      processing_mode  = "GoTemplate"
      tokens = {
        DefaultDatabaseId = "11111111-1111-1111-1111-111111111111"
      }
    }
  }
}
8 Β· Explicit Parameters processing mode (JsonPathReplace)

πŸ”’ processing_mode defaults to "None" at the module level (overriding the provider's own "GoTemplate" default) per this module suite's conventions Β§ Secure-by-default β€” this is a deliberate opt-in.

module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  format       = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source          = "${path.module}/files/RealTimeQueryset.json.tmpl"
      processing_mode = "Parameters"
      parameters = [
        {
          type  = "JsonPathReplace"
          find  = "$.properties.defaultDatabaseId"
          value = "22222222-2222-2222-2222-222222222222"
        }
      ]
    }
  }
}
9 Β· TextReplace parameter substitution
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id
  format       = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source          = "${path.module}/files/RealTimeQueryset.json.tmpl"
      processing_mode = "Parameters"
      parameters = [
        {
          type  = "TextReplace"
          find  = "__ENVIRONMENT__"
          value = "prod"
        }
      ]
    }
  }
}
10 Β· Custom timeouts
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-telemetry-ops-prod"
  workspace_id = module.workspace.id

  timeouts = {
    create = "20m"
    update = "10m"
  }
}
11 Β· for_each fan-out across a medallion-style set of querysets
locals {
  querysets = {
    ops     = "kqs-ops-metrics-prod"
    claims  = "kqs-claims-triage-prod"
    finance = "kqs-finance-recon-prod"
  }
}

module "kql_queryset" {
  source   = "../../"
  for_each = local.querysets

  display_name = each.value
  workspace_id = module.workspace.id
}
12 Β· Least-privilege reviewer scenario (Viewer cannot write)

ℹ️ A caller whose SPN/user only holds the workspace Viewer role will see apply fail on this resource β€” Viewer is explicitly excluded from the "write or delete KQL Querysets" capability. See πŸ” Troubleshooting.

# NOT a working example β€” illustrates the failure mode described above.
# module "kql_queryset" {
# source = "../../"
# display_name = "kqs-readonly-attempt"
# workspace_id = module.workspace.id # workspace where caller only holds Viewer
# }
13 Β· Governed folder nesting (raw/curated split)
module "kql_queryset" {
  source = "../../"

  display_name = "kqs-curated-metrics-prod"
  workspace_id = module.workspace.id
  folder_id    = module.folder_curated.id
}
14 Β· Explicit format without a definition (no-op)

ℹ️ format is only meaningful when definition is populated; setting it alone with an empty definition has no observable effect but is accepted by validate.

module "kql_queryset" {
  source = "../../"

  display_name = "kqs-placeholder-dev"
  workspace_id = module.workspace.id
  format       = "Default"
}
15 Β· πŸ—οΈ End-to-end composition

Wires a workspace, a folder, a tag, and this module together β€” the composition a real curated telemetry queryset looks like in this library.

module "workspace" {
  source = "git::https://github.com/microsoftexpert/terraform-fabric-workspace.git?ref=v1.0.0"

  display_name = "ws-observability-core-prod"
  description  = "Core observability workspace β€” prod. Owner: Observability."
}

module "folder_curated" {
  source = "git::https://github.com/microsoftexpert/terraform-fabric-folder.git?ref=v1.0.0"

  workspace_id = module.workspace.id
  display_name = "curated"
}

resource "fabric_tag" "observability" {
  display_name = "observability"
}

module "kql_queryset" {
  source = "git::https://github.com/microsoftexpert/terraform-fabric-kql-queryset.git?ref=v1.0.0"

  display_name = "kqs-telemetry-ops-prod"
  description  = "Curated telemetry queryset β€” prod. Owner: Observability."
  workspace_id = module.workspace.id
  folder_id    = module.folder_curated.id
  format       = "Default"

  definition = {
    "RealTimeQueryset.json" = {
      source          = "${path.module}/files/RealTimeQueryset.json.tmpl"
      processing_mode = "GoTemplate"
      tokens = {
        DefaultDatabaseId = "33333333-3333-3333-3333-333333333333"
      }
    }
  }

  tags = [fabric_tag.observability.id]
}

πŸ“₯ Inputs

Variable Type Default Notes
display_name string β€” (required) Non-empty
workspace_id string β€” (required) Non-empty
description string null
folder_id string null Workspace-root placement if omitted
format string null Only legal value: "Default"
definition map(object({...})) {} Keyed by "RealTimeQueryset.json" for format = "Default"
definition_update_enabled bool true Matches provider default
tags set(string) [] fabric_tag GUIDs
timeouts object({...}) null
Full object schemas
variable "definition" {
  type = map(object({
    source           = string
    tokens_delimiter = optional(string, "{{}}")
    tokens           = optional(map(string), {})
    processing_mode  = optional(string, "None")
    parameters = optional(set(object({
      find  = string
      type  = string
      value = string
    })), [])
  }))
  default = {}
}

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

🧾 Outputs

Output Description Sensitive
id KQL Queryset GUID β€”
display_name KQL Queryset display name β€”

🧠 Architecture Notes

  • definition and timeouts are assigned directly with = in main.tf β€” never a dynamic block β€” since the live v1.12.0 schema documents both as Terraform Plugin Framework Attributes types, not classic SDKv2 Blocks. This mirrors the resolution already established by terraform-fabric-lakehouse, terraform-fabric-sql-database, and terraform-fabric-workspace elsewhere in this library.
  • The live schema does not expose a top-level reference to a target KQL database/Eventhouse. A Queryset's default database context lives inside the definition JSON payload itself (e.g. a defaultDatabaseId field), typically injected via parameters (JsonPathReplace against $.properties.defaultDatabaseId) rather than as a separate Terraform argument. Do not add a kql_database_id-shaped variable to this module β€” it would map to nothing in the schema.
  • definition currently accepts exactly one part key ("RealTimeQueryset.json", paired with format = "Default") β€” still modeled as a generic map(object(...)) rather than a single nested object, so the module doesn't hard-code an assumption that the provider will never add a second accepted format.
  • processing_mode's module-level default ("None") deliberately overrides the provider's own "GoTemplate" default per this module suite's conventions Β§ Secure-by-default β€” no implicit token substitution without an explicit caller opt-in.

🧱 Design Principles

Concern Safe default Opt-out
Folder placement folder_id = null β€” workspace root Caller supplies folder_id
Definition content definition = {} β€” no predefined content Caller supplies a definition part
Template substitution processing_mode = "None" (module default; provider default is "GoTemplate") Caller sets "GoTemplate" or "Parameters" explicitly
Tag assignment tags = [] Caller supplies fabric_tag GUIDs

πŸš€ Runbook

cd C:\GitHubCode\newfabricmodules\terraform-fabric-kql-queryset
terraform init -backend=false
terraform validate
terraform fmt -check

Pin consumers to ?ref=v1.0.0 β€” never a branch. This module is plan-only; a human applies from a reviewed, approved CI pipeline.


πŸ§ͺ Testing

  • terraform validate catches: a malformed definition entry (missing source, illegal tokens_delimiter/processing_mode/parameters[].type value), an illegal format value, an empty display_name/workspace_id.
  • terraform fmt -check catches formatting drift without silently rewriting files.
  • Not caught offline: whether the calling principal actually holds a workspace role permitting KQL Queryset writes, whether the target capacity is Active, whether the tenant's "Service principals can call Fabric public APIs" setting is enabled, or whether a definition payload's embedded database reference actually resolves to a real KQL database. These require a live plan/apply against a real tenant.

πŸ’¬ Example Output

module.kql_queryset.id = "4a7b2c1d-9e3f-4a8b-b2f1-6c5d8e9f0a1b"
module.kql_queryset.display_name = "kqs-telemetry-ops-prod"

πŸ” Troubleshooting

Symptom Cause Fix
apply fails with a 403/permission error Calling principal holds only Viewer (or no) role on the workspace Grant Admin, Member, or Contributor on the workspace
Entire module fails on every resource, every environment Tenant-level "Service principals can call Fabric public APIs" setting not enabled Have a Fabric administrator flip the Developer setting in the Fabric Admin Portal
definition silently doesn't update after changing the source file definition_update_enabled = false Set it to true (the default) if drift tracking is desired
Token placeholders appear literally in the created Queryset instead of substituted processing_mode left at the module's secure default ("None") Explicitly set processing_mode = "GoTemplate" or "Parameters"
Queryset created but shows no data / errors when run definition's embedded database reference doesn't resolve, or the caller lacks read/editing permission on the target KQL database Confirm the database GUID and the caller's permissions on that database (out of this module's scope)

πŸ”— Related Docs

Releases

Packages

Contributors

Languages