Registers an Azure Machine Learning datastore pointing at a blob container, defaulting to credential-free identity-based access, targeting
hashicorp/azurerm ~> 4.0.
- βοΈ Creates one
azurerm_machine_learning_datastore_blobstorage, namedthis. - π Defaults to identity-based access, where the provider defaults to credential-based. The provider's default
caches a storage key in the workspace's Key Vault β retrievable by anyone with workspace
Reader. - π― Groups the three credential fields into one variable so both of Microsoft's documented rules become parse-time errors: the key/SAS conflict, and the credential-required-unless-identity rule.
- π Emits
datastore_uri_prefixβ theazureml://string a notebook actually uses, which exists nowhere in the resource's own state. - π€ Emits
storage_account_id, derived by truncating the container ID, ready to be a role-assignment scope. β οΈ Rejects a bare workspace GUID by name, because the workspace module's correctly-namedworkspace_idoutput is the wrong one.
π‘ Why it matters: a datastore looks like a two-line resource and is really a choice about where a storage key lives. Credential-based access puts an account key β full control of every container in the account β behind a workspace
Readerrole that people hand out for visibility. Identity-based access stores nothing. The provider defaults to the first; this module defaults to the second and is explicit about the one prerequisite that makes it work.
If this module saves you time:
- β Star the repository β it genuinely helps others find it.
- πΌ Connect on LinkedIn β linkedin.com/in/microsoftexpert
- β Buy me a coffee β buymeacoffee.com/microsoftexpert
flowchart TB
RG["terraform-azurerm-resource-group"]
SA["terraform-azurerm-storage-account -- the data this references. NEVER created or deleted by a datastore."]
MLW["terraform-azurerm-machine-learning-workspace"]
BLOB["terraform-azurerm-machine-learning-datastore-blobstorage"]
FS["terraform-azurerm-machine-learning-datastore-fileshare"]
ADLS["terraform-azurerm-machine-learning-datastore-datalake-gen2"]
RA["terraform-azurerm-role-assignments -- Storage Blob Data Reader, required for identity-based access"]
CC["terraform-azurerm-machine-learning-compute-cluster -- runs the jobs that read these datastores"]
RG -->|"name as resource_group_name"| MLW
MLW -->|"id as workspace_id -- NOT its workspace_id output, which is a GUID"| BLOB
MLW -->|"id as workspace_id"| FS
MLW -->|"id as workspace_id"| ADLS
SA -->|"container id as storage_container_id"| BLOB
SA -->|"share id as storage_fileshare_id -- A DIFFERENT ARGUMENT NAME"| FS
SA -->|"container id as storage_container_id"| ADLS
MLW -->|"identity_principal_id"| RA
BLOB -->|"storage_account_id as the role scope"| RA
BLOB -->|"datastore_uri_prefix"| CC
style BLOB fill:#0078D4,stroke:#004578,color:#ffffff
style MLW fill:#004578,stroke:#00335c,color:#ffffff
style RG fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style SA fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style FS fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style ADLS fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style RA fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style CC fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
Two edges are the ones to get right. The workspace's id becomes workspace_id β not its workspace_id output,
which is a GUID. And the role assignment is not optional decoration: identity-based access is the default here, and it
does nothing until the workspace identity holds Storage Blob Data Reader on the account. This DAG is shared with the
two sibling datastore modules, which differ in their storage target and their authentication.
flowchart TB
REQ["required: name, workspace_id, storage_container_id"]
CRED["credential -- ONE variable holding account_key, shared_access_signature and service_data_auth_identity, because TWO documented rules span them. Marked sensitive as a whole."]
R1["RULE 1: account_key CONFLICTS WITH shared_access_signature. At most one."]
R2["RULE 2: if service_data_auth_identity is None or omitted, one of the two credentials is REQUIRED"]
FLIP["THIS MODULE DEFAULTS service_data_auth_identity TO WorkspaceSystemAssignedIdentity. The provider defaults it to None, which caches a storage key in the workspace Key Vault."]
THIS["azurerm_machine_learning_datastore_blobstorage.this"]
KV["workspace Key Vault -- caches the key or SAS when credential-based. ANY WORKSPACE READER CAN RETRIEVE IT."]
ROLE["identity-based instead needs Storage Blob Data Reader on the storage account. Nothing here can check that."]
URI["derived: datastore_uri_prefix, the azureml:// string a notebook actually references"]
DERIVED["derived: authentication_mode, stores_no_credential_in_the_workspace_key_vault, storage_account_id, posture_summary"]
REQ --> THIS
CRED --> R1
CRED --> R2
CRED --> FLIP
CRED --> THIS
THIS -->|"credential-based only"| KV
THIS -->|"identity-based"| ROLE
THIS --> URI
THIS --> DERIVED
style THIS fill:#0078D4,stroke:#004578,color:#ffffff
style KV fill:#004578,stroke:#00335c,color:#ffffff
style REQ fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style CRED fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style R1 fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style R2 fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style FLIP fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style ROLE fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style URI fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
style DERIVED fill:#f2f4f7,stroke:#98a2b3,color:#1d2939
| Resource | Count | Notes |
|---|---|---|
azurerm_machine_learning_datastore_blobstorage.this |
1 | The keystone. |
nested timeouts block |
0 or 1 | All four operations (30m/5m/30m/30m). |
| Requirement | Value |
|---|---|
| Terraform | >= 1.12.0 |
hashicorp/azurerm |
~> 4.0 |
| Provider block | None here. The caller configures the provider, its authentication, and the mandatory features {} block. |
| Resource | azurerm_machine_learning_datastore_blobstorage |
| ARM type | Microsoft.MachineLearningServices/workspaces/dataStores |
| API provider | Microsoft.MachineLearningServices, version 2025-06-01 |
- π΄ The workspace module's
workspace_idoutput is NOT what this argument wants. The workspace emitsid(the ARM Resource ID) andworkspace_id(an immutable GUID). This argument is namedworkspace_id, soworkspace_id = module.mlw.workspace_idreads perfectly and is wrong. Passmodule.mlw.id. A bare GUID is rejected here with that message. - π΄
account_keyconflicts withshared_access_signature, and "ifservice_data_auth_identityis set toNoneor omitted, one ofaccount_keyorshared_access_signaturemust be specified." Two rules, three fields β hence one variable. - π΄ A credential-based datastore's key is readable by workspace
Readers. Microsoft: the credential is cached in the workspace's Key Vault and "users with Reader workspace access can access the credentials." - π΄
tagsanddescriptionare FORCE-NEW. Unusual enough to be worth repeating: a routine retagging sweep destroys and recreates every datastore it touches. Updatable in place: the credentials, the identity,is_default. - π΄
is_defaultcan only be set totrueon update. A first apply withtruefails, and novalidation {}can catch it β a variable validation cannot see create-versus-update. β οΈ The argument isservice_data_auth_identityhere andservice_data_identityon both siblings. Two spellings of one concept across three resources.β οΈ is_defaultis settable here and read-only (computed) on both siblings.β οΈ The identity values carry aWorkspaceprefix βWorkspaceSystemAssignedIdentity, not theSystemAssignedof a normal ARMidentityblock. One prefix apart, different mechanism; the wrong vocabulary is rejected by name.β οΈ The ARM segment isdataStores, capital S.β οΈ A datastore URI uses lower-caseresourcegroupswhere an ARM ID usesresourceGroupsβ which is whydatastore_uri_prefixis emitted rather than assembled by hand.- βΉοΈ A SAS expires and nothing in Terraform tracks it. It works until it doesn't, with no plan showing a change.
Force-new facts come from the provider's documentation markdown. The binary schema carries no force-new marker at all. Shapes, types and sensitivity marks come from the schema; the credential-visibility and role requirements come from Microsoft's platform documentation.
| Role | Scope | Why |
|---|---|---|
Contributor or AzureML Data Scientist |
the ML workspace | Create, update, delete the datastore. |
Reader |
the ML workspace | Refresh and plan β but see the warning. |
Storage Blob Data Reader |
the storage account | Granted to the workspace's identity, not Terraform's. Required for identity-based access; unnecessary on the workspace's own default account. |
Reader on the storage private endpoint |
that endpoint | Only when the account is behind a private endpoint. Also the workspace identity. |
| (none) storage permission for Terraform | β | The Terraform principal never touches the data plane. |
π΄
Readeron the workspace is not a read-only role when credential-based datastores exist. Microsoft names it as a limitation: the credential is cached in the workspace Key Vault and "other workspace users with sufficient permissions can retrieve those credentials."The interesting direction is the reverse of how it sounds.
Readeris the role granted freely for visibility β and on such a workspace it grants the storage account key, which controls every container in that account, not just this one. No storage-side permission limits it, because the holder is not using one. Put it in the workspace's access review. Or store no credential, which is the default here.
- An existing Machine Learning workspace, and an existing storage account and blob container. A datastore is a reference; this module creates none of them.
Microsoft.MachineLearningServicesregistered on the subscription.- For identity-based access (the default):
Storage Blob Data Readerfor the workspace identity on the storage account β except on the workspace's own default account. The prerequisite most likely to be missed, because its absence produces no Terraform error at all. - Behind a private endpoint: the workspace identity also needs
Readeron the endpoint. An account may have separate blob, file and dfs endpoints. - For user-identity passthrough on a credential-less blob datastore: soft delete must not be enabled. A published constraint, invisible from here.
- To set
is_default = true: an existing datastore, since the provider permits it only on update.
terraform-azurerm-machine-learning-datastore-blobstorage/
βββ providers.tf # required_version + the pinned azurerm; no provider block
βββ variables.tf # 8 inputs, 18 validations
βββ main.tf # locals deriving the URI, the account ID and the auth posture
βββ outputs.tf # 24 outputs: id first, then the derived posture, then the constants
βββ README.md # this file
βββ SCOPE.md # the cross-module contract
βββ LICENSE # MIT
βββ .gitignore # the canonical library ignore file
provider "azurerm" {
features {}
}
module "training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
name = "training_data"
workspace_id = module.mlw.id # NOT module.mlw.workspace_id
storage_container_id = var.training_container_id
}Three arguments. No credential β the module defaults to WorkspaceSystemAssignedIdentity, so nothing is stored in the
workspace's Key Vault.
β οΈ That empty call needs one thing outside it:Storage Blob Data Readerfor the workspace identity on the storage account. Without it the datastore is created cleanly and jobs fail later. See example 3.
| Input | Type | Typical source |
|---|---|---|
workspace_id |
string |
module.mlw.id β not module.mlw.workspace_id |
storage_container_id |
string |
var.container_id |
name |
string |
caller |
credential |
object, sensitive |
omit it; or a secret store, out of band |
description, is_default, tags, timeouts |
β | caller |
| Output | Consumed by |
|---|---|
id |
anything referencing the datastore |
datastore_uri_prefix |
job definitions, data assets, notebooks |
storage_account_id |
terraform-azurerm-role-assignments β scope |
uses_identity_based_access, stores_no_credential_in_the_workspace_key_vault |
check blocks, security review |
posture_summary |
a composition's own output |
The examples below reference existing resources by ID or name rather than creating them; this module owns only its own resource. Those references are declared inputs:
variable "container_id" {
description = "id of an existing container that these examples reference but do not create."
type = string
}
variable "training_container_id" {
description = "id of an existing training container that these examples reference but do not create."
type = string
}1 Β· The minimum call β credential-free
module "training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
name = "training_data"
workspace_id = module.mlw.id
storage_container_id = var.training_container_id
}π‘
authentication_modereadsIdentity-based,supplied_credential_countreads0, andstores_no_credential_in_the_workspace_key_vaultreadstrue. The provider would have given youNoneβ meaning credential-based β and then rejected the call for having no credential. This module's default is both safer and the shorter thing to type.
2 Β· The mistake that reads perfectly
# FAILS AT PARSE TIME, and the message says why.
workspace_id = module.mlw.workspace_id
# Correct.
workspace_id = module.mlw.idπ΄ The workspace module emits both.
idis the ARM Resource ID;workspace_idis the workspace's immutable GUID, a different thing entirely. Because this argument is also calledworkspace_id, the wrong one is the one whose name matches β the single most seductive wiring error in this family. The module rejects a bare GUID and names the output to use instead.
3 Β· π The role assignment identity-based access needs
module "training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
name = "training_data"
workspace_id = module.mlw.id
storage_container_id = var.training_container_id
}
module "mlw_can_read_training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"
# Derived by truncating the container ID -- no need to restate the account.
scope = module.training_data.storage_account_id
role_assignments = {
blob_read = {
role_definition_name = "Storage Blob Data Reader"
principal_id = module.mlw.identity_principal_id
principal_type = "ServicePrincipal"
}
}
}π‘ This is the failure mode worth internalising. Without the grant, the datastore is created successfully, the plan is clean, and a job reading
azureml://.../datastores/training_data/paths/x.csvfails days later with an authorization error against the storage account. Nothing in Terraform ever mentions it.βΉοΈ Microsoft documents one exception: the grant is unnecessary when this is the workspace's own default storage account.
4 Β· Credential-based, when you must
data "azurerm_storage_account" "legacy" {
name = "legacydatastore"
resource_group_name = "rg-legacy"
}
module "legacy_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
name = "legacy_data"
workspace_id = module.mlw.id
storage_container_id = "${data.azurerm_storage_account.legacy.id}/blobServices/default/containers/exports"
credential = {
# BOTH fields are needed: the identity must be explicitly "None".
service_data_auth_identity = "None"
account_key = data.azurerm_storage_account.legacy.primary_access_key
}
}π Three consequences.
sensitive = trueredacts plan output and does not encrypt state β that key is in your state file in plaintext. The key is also cached in the workspace's Key Vault, where any workspaceReadercan retrieve it. And an account key grants full control of every container in that account, not justexports.π‘
supplied_credential_countreads1and the module never emits which field it was, becauseaccount_keyandshared_access_signaturedescribe very different blast radii.
5 Β· A SAS instead of an account key
credential = {
service_data_auth_identity = "None"
shared_access_signature = var.exports_container_sas
}π‘ Narrower than an account key and still a stored credential β prefer it if a credential is unavoidable.
β οΈ A SAS expires and nothing in Terraform tracks it. The datastore keeps working until the expiry passes, then fails at job time with no plan showing any change.uses_sas_tokenreadstrue; put the expiry in a calendar.
6 Β· The two SAS formats the portal offers
# BOTH FAIL AT PARSE TIME.
credential = {
service_data_auth_identity = "None"
shared_access_signature = "?sv=2022-11-02&ss=b&sig=REDACTED"
}
credential = {
service_data_auth_identity = "None"
shared_access_signature = "https://acct.blob.core.windows.net/exports?sv=2022-11-02&sig=REDACTED"
}
# Correct -- the token alone.
credential = {
service_data_auth_identity = "None"
shared_access_signature = "sv=2022-11-02&ss=b&srt=sco&sp=rl&sig=REDACTED"
}βΉοΈ The portal shows a SAS twice: as a token beginning
?sv=and as a full Blob SAS URL. This argument wants neither of those verbatim β it wants the token with the leading?stripped. Both wrong forms are rejected by name, because copying from the portal is the normal way to obtain one.
7 Β· What you cannot do β both credentials
# FAILS AT PARSE TIME.
credential = {
service_data_auth_identity = "None"
account_key = var.key
shared_access_signature = var.sas
}
β οΈ The provider documents them as conflicting. They are two alternative credentials for the same container, and supplying both leaves which one Azure uses undefined. This is one of the two rules that only became enforceable by putting the three fields in a single variable.
8 Β· And what you cannot do β no credential with `None`
# FAILS AT PARSE TIME.
credential = {
service_data_auth_identity = "None"
}
β οΈ The provider's rule: "ifservice_data_auth_identityis set toNoneor omitted, one ofaccount_keyorshared_access_signaturemust be specified." SoNoneis a commitment to supplying a credential.π‘ The error message names the way out: set
service_data_auth_identitytoWorkspaceSystemAssignedIdentityβ which is the module's default β and supply nothing.
9 Β· The wrong identity vocabulary
# FAILS -- this is the ARM identity-block vocabulary, a different mechanism.
credential = { service_data_auth_identity = "SystemAssigned" }
# FAILS -- recognised value, wrong case.
credential = { service_data_auth_identity = "workspacesystemassignedidentity" }
# Correct.
credential = { service_data_auth_identity = "WorkspaceSystemAssignedIdentity" }π‘
SystemAssignedandUserAssignedare what anidentity {}block takes on most Azure resources. This argument selects which of the workspace's identities retrieves data, so the values carry aWorkspaceprefix. One prefix apart, and a caller who typedSystemAssignedwas not guessing β they were applying a pattern that holds everywhere else. The message maps it across.
10 Β· A user-assigned workspace identity
credential = {
service_data_auth_identity = "WorkspaceUserAssignedIdentity"
}βΉοΈ Also credential-free. The workspace must already have a user-assigned identity attached β that is the workspace module's
identityinput, not this module's concern β and that identity is the one needingStorage Blob Data Reader.uses_identity_based_accessreadstrueeither way; the distinction is which principal to grant.
11 Β· A credential alongside an identity β legal, pointless, reported
credential = {
service_data_auth_identity = "WorkspaceSystemAssignedIdentity"
shared_access_signature = var.exports_container_sas
}
β οΈ credential_supplied_but_identity_selectedreadstrue. This is accepted β the provider documents no conflict, so rejecting it could refuse input the API accepts β and the SAS has no effect, because the identity is what Azure uses to retrieve data. The cost is a pointless credential sitting in the workspace's Key Vault and in your state file, retrievable by any workspaceReader. Remove it.π‘ This is the difference between reject and report: a caller could plausibly have meant this as a transition step, so the module says so rather than refusing.
12 Β· The `is_default` rule Terraform cannot enforce
# Apply ONE: creates the datastore.
module "training_data" {
# ...
is_default = false
}
# Apply TWO: promotes it. Only now is `true` accepted.
module "training_data" {
# ...
is_default = true
}
β οΈ The provider documents it in one line: "is_defaultcan only be set totrueon update." A first apply withtruefails at Azure.π‘ No
validation {}block can catch this, because a variable validation runs against the input value alone and cannot see whether Terraform is creating or updating. It is the only rule in this module that is both published and genuinely uncheckable, which is why it is a constant output instead:is_default_cannot_be_set_to_true_at_creation.
13 Β· Rejecting the wrong storage target
# FAILS -- a FILE SHARE id. The sibling module wants this, under a different argument name.
storage_container_id = "${data.azurerm_storage_account.x.id}/fileServices/default/shares/share1"
# FAILS -- a bare STORAGE ACCOUNT id. Nothing in it says which container.
storage_container_id = data.azurerm_storage_account.x.id
# Correct.
storage_container_id = "${data.azurerm_storage_account.x.id}/blobServices/default/containers/exports"π The regex is anchored at both ends. A file share lives under
/fileServices/default/shares/and a blob container under/blobServices/default/containers/β one path segment apart, and they belong to two different Terraform resources. The message names the sibling module and its differently-named argument.
14 Β· Several datastores on one workspace
locals {
datastores = {
training_data = "training-data"
validation = "validation-data"
features = "feature-store"
}
}
module "datastores" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
for_each = local.datastores
name = each.key
workspace_id = module.mlw.id
storage_container_id = "${module.data_lake.id}/blobServices/default/containers/${each.value}"
}
output "datastore_uris" {
value = { for k, m in module.datastores : k => m.datastore_uri_prefix }
}π‘
for_eachover a keyed map, nevercountβ the key is the datastore name, so adding a fourth does not re-index the first three. And because all three point into one storage account, oneStorage Blob Data Readerassignment at the account scope covers them all.
β οΈ Notenameis force-new and appears inside everyazureml://URI. Changing a map key here recreates the datastore and breaks every notebook referencing the old name.
15 Β· ποΈ End-to-end composition
provider "azurerm" {
features {}
}
locals {
platform_tags = {
environment = "production"
owner = "data-platform"
}
}
module "ml_rg" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"
name = "rg-ml-eastus"
location = "eastus"
tags = local.platform_tags
}
module "data_lake" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-storage-account.git?ref=v1.0.0"
name = "stmlplatformeastus"
resource_group_name = module.ml_rg.name
location = module.ml_rg.location
account_tier = "Standard"
account_replication_type = "ZRS"
containers = {
training-data = {}
}
tags = local.platform_tags
}
module "mlw" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-workspace.git?ref=v1.0.0"
name = "mlw-platform-eastus"
resource_group_name = module.ml_rg.name
location = module.ml_rg.location
application_insights_id = var.platform_app_insights_id
key_vault_id = var.platform_key_vault_id
storage_account_id = module.data_lake.id
identity = { type = "SystemAssigned" }
tags = local.platform_tags
}
module "training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-machine-learning-datastore-blobstorage.git?ref=v1.0.0"
name = "training_data"
# The workspace's ARM id -- NOT its workspace_id output, which is a GUID.
workspace_id = module.mlw.id
storage_container_id = "${module.data_lake.id}/blobServices/default/containers/training-data"
# Omitted entirely: the module defaults to WorkspaceSystemAssignedIdentity,
# so no key is cached in the workspace Key Vault and none reaches state.
description = "Curated training data for the platform models."
# tags are FORCE-NEW on this resource -- set once, and keep changeable
# metadata on the workspace or the storage account instead.
tags = local.platform_tags
}
# WITHOUT THIS the datastore is created cleanly and every job reading it fails.
module "mlw_reads_training_data" {
source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"
scope = module.training_data.storage_account_id
role_assignments = {
blob_read = {
role_definition_name = "Storage Blob Data Reader"
principal_id = module.mlw.identity_principal_id
principal_type = "ServicePrincipal"
}
}
}
check "datastore_posture" {
assert {
condition = module.training_data.uses_identity_based_access
error_message = "Datastores must use identity-based access, not a cached storage credential."
}
assert {
condition = module.training_data.stores_no_credential_in_the_workspace_key_vault
error_message = "A credential in the workspace Key Vault is retrievable by any workspace Reader."
}
assert {
condition = !module.training_data.credential_supplied_but_identity_selected
error_message = "A credential was supplied alongside an identity. It has no effect -- remove it."
}
}
output "training_datastore" {
value = {
id = module.training_data.id
uri = module.training_data.datastore_uri_prefix
summary = module.training_data.posture_summary
account = module.training_data.storage_account_name
# Terraform confirms the reference exists. It cannot confirm the identity can read.
read_access_confirmed_by_terraform = false
}
}π‘ Four things this composition gets right. The workspace's
idβ not itsworkspace_idβ is wired in. The credential is omitted entirely rather than sourced and protected. The role assignment's scope ismodule.training_data.storage_account_id, derived from the container ID rather than restated. And thecheckblock asserts the posture instead of assuming it.
β οΈ Noteread_access_confirmed_by_terraform = false, hard-coded. A clean apply proves the datastore reference exists. It proves nothing about whether the workspace identity can actually read the container β that fails later, in a job, with nothing in Terraform to point at.
Required (3): name, workspace_id, storage_container_id β all force-new.
The grouped credential (1): credential, holding account_key, shared_access_signature and
service_data_auth_identity. Marked sensitive. Defaults to identity-based.
Optional (4): description (force-new), is_default, tags (force-new), timeouts.
Full input schemas
variable "name" {
type = string
# Force-new. Appears inside every azureml:// URI referencing this datastore.
}
variable "workspace_id" {
type = string
# Force-new. Anchored to .../Microsoft.MachineLearningServices/workspaces/<name>.
# REJECTS A BARE GUID -- that is the workspace's OTHER id, emitted by the workspace
# module as `workspace_id`. This argument wants `module.<workspace>.id`.
}
variable "storage_container_id" {
type = string
# Force-new. Anchored to .../blobServices/default/containers/<name>.
# Rejects a file share ID (the sibling module's target) and a bare account ID.
}
variable "credential" {
type = object({
account_key = optional(string)
shared_access_signature = optional(string)
service_data_auth_identity = optional(string) # None | WorkspaceSystemAssignedIdentity | WorkspaceUserAssignedIdentity
})
default = {
service_data_auth_identity = "WorkspaceSystemAssignedIdentity"
}
sensitive = true
# TWO documented rules, both enforceable only because the fields share a variable:
# 1. account_key CONFLICTS WITH shared_access_signature.
# 2. identity "None" (or omitted) REQUIRES one of the two.
# THE PROVIDER DEFAULTS THE IDENTITY TO "None" (credential-based). This module does not.
}
variable "description" {
type = string
default = null
# FORCE-NEW, unusually for a description.
}
variable "is_default" {
type = bool
default = false
# CANNOT BE `true` AT CREATION -- provider rule, and unenforceable in a validation.
# Settable here; READ-ONLY on both sibling datastore resources.
}
variable "tags" {
type = map(string)
default = {}
# FORCE-NEW. A retagging sweep recreates the datastore.
}
variable "timeouts" {
type = object({
create = optional(string)
read = optional(string)
update = optional(string)
delete = optional(string)
})
default = null
# Provider defaults: 30m / 5m / 30m / 30m.
}
β οΈ Object-type conversion silently discards extra keys. A misspelled key insidecredentialortimeoutsis dropped with no error at all β so a typo'dservice_data_identity(the sibling's spelling) would be silently ignored here, leaving the provider'sNonedefault in force. Checkservice_data_auth_identity_effectiveagainst what you intended.
| Output | Description |
|---|---|
id |
The datastore's Resource ID. Note dataStores, capital S. |
name |
The datastore name, as it appears in every azureml:// URI. |
datastore_uri_prefix |
π The azureml:// prefix workloads use. Assembled from four values; in state nowhere. |
workspace_id / workspace_name / workspace_resource_group / workspace_subscription_id |
The parent, echoed and decomposed. |
storage_container_id / storage_container_name |
The container referenced. |
storage_account_id |
Derived by truncating the container ID β a ready-made role-assignment scope. |
storage_account_name / storage_account_resource_group |
Which account this actually points at. |
authentication_mode |
Identity-based or Credential-based, in Microsoft's own words. |
service_data_auth_identity_effective |
The identity in force, with the provider's None default applied. |
uses_identity_based_access |
Derived β true on an empty call, because of the flip. |
stores_no_credential_in_the_workspace_key_vault |
The goal, stated positively. |
uses_account_key / uses_sas_token |
Which kind, if any. An account key controls the whole account. |
supplied_credential_count |
π A count. Never a value, never a field name. |
credential_supplied_but_identity_selected |
Legal, pointless, reported. |
is_default |
Settable here; read-only on both siblings. |
posture_summary |
The configuration as one reviewable line. |
identity_based_access_needs_a_role_assignment |
Constant. Failure-timing table with two Never rows. |
credential_based_access_is_readable_by_workspace_readers |
Constant. The RBAC asymmetry. |
is_default_cannot_be_set_to_true_at_creation |
Constant. Published and uncheckable. |
a_datastore_is_a_reference_and_deleting_it_deletes_no_data |
Constant. And what destroying it does break. |
tags_and_description_are_force_new_on_this_resource |
Constant. |
the_arm_type_is_a_child_of_the_workspace |
The ARM type string. |
secure_by_default_applies_here_and_this_is_how |
Constant. The flip, and why it was safe to make. |
No credential is emitted in any form, including a field name.
A datastore is a reference, and the only real decision is where the key lives. Microsoft offers two mechanisms.
Credential-based access caches an account key or SAS in the workspace's Key Vault β and Microsoft names the consequence
as a limitation: "users with Reader workspace access can access the credentials." Identity-based access stores nothing;
"you don't save any authentication credentials. You only store the storage account information." The provider defaults
to the first. This module defaults to the second, and uses_identity_based_access reads true on an empty call so
the divergence is visible rather than surprising.
The flip was checked against the at-least-one-of rule before being made, and that check is the interesting part.
A default that silently satisfies an at-least-one-of requirement removes a legal shape β which is exactly why a
comparable flip was declined on another resource in this suite. Here it does not: a caller wanting credential-based
access still writes service_data_auth_identity = "None" alongside a key or SAS, and nothing becomes unreachable. The
flip only changes which shape you get for free.
Two documented rules span three fields, so the three fields are one variable. A validation {} block may reference
only its own variable, so grouping account_key, shared_access_signature and service_data_auth_identity is what
makes both rules parse-time errors: the conflict between the two credentials, and the requirement that one of them exist
when the identity is None. Note the second is a conditional requirement rather than a plain exclusion β the
credential-free shape is valid only when an identity is selected β which is a case grouping handles as neatly as a
simple XOR.
Sensitivity is handled at the variable, because that is the only place it can be. sensitive applies to a variable
and not to an attribute, so the whole credential object carries the mark even though service_data_auth_identity is
not a secret. The consequence is contagion: try(var.credential.account_key, null) != null is itself a sensitive bool
that Terraform refuses to emit, so every derived fact is unwrapped with nonsensitive() β which doubles as a guard,
since it errors if its argument was not sensitive. What is deliberately not unwrapped is any field name: account_key
versus shared_access_signature describe very different blast radii, so the kind of credential is itself disclosure.
The count is what a review needs.
The most consequential thing this module cannot check is a role assignment. Identity-based access needs the
workspace's managed identity to hold Storage Blob Data Reader on the storage account β except on the workspace's own
default account, where Microsoft says it is unnecessary. Without it, the datastore is created cleanly, the plan is
clean, and a job fails days later with an authorization error against storage. So
identity_based_access_needs_a_role_assignment is a constant with a failure-timing table whose decisive rows read
Never, and storage_account_id is derived from the container ID specifically so the composition can wire the grant
without restating the account.
That derivation is a place to be careful, because getting it wrong is silent. The account name sits five segments
back from the end of a container ID, not four β an off-by-one yields blobServices as the "storage account name", which
validates, formats and reads plausibly in a summary string. The index arithmetic is commented with the segment count for
that reason.
datastore_uri_prefix exists because the value that gets used is not in the resource's state. A notebook references
azureml://subscriptions/β¦/resourcegroups/β¦/workspaces/β¦/datastores/<name>/paths/β¦, assembled from four separate
values β and note resourcegroups is lower case and unhyphenated there, where an ARM Resource ID uses resourceGroups.
That inconsistency is Azure's, and it produces URIs that look right and resolve to nothing, so the module builds the
prefix rather than leaving it to be retyped.
Two force-new fields are surprising enough to be a constant. tags and description both force replacement on this
resource, where almost every other resource in this library updates them in place. A subscription-wide retagging sweep
therefore destroys and recreates every datastore it touches. What is updatable in place: the credentials, the identity
and is_default β so rotating a credential is cheap and renaming is not.
One published rule is genuinely uncheckable, and that is stated rather than worked around. is_default can only be
set to true on update. A variable validation runs against an input value with no knowledge of prior state, so it cannot
see create-versus-update. The module reports the rule instead of pretending to enforce it.
The three sibling datastores are not interchangeable, and the differences were re-derived rather than copied. Two
spellings of the identity argument (service_data_auth_identity here, service_data_identity on both siblings); two
lifecycles for is_default (settable here, computed there); two authentication mechanisms (key or SAS here and on
fileshare, a service principal on Data Lake Gen2); two storage-target arguments (storage_container_id here and on
Data Lake Gen2, storage_fileshare_id on fileshare); and one rule the provider documents on this resource only. Reading
one member of the trio and generalising would have produced three subtly wrong modules.
for_each, never count where several datastores share a workspace β and because they usually share a storage
account too, one account-scoped role assignment covers all of them.
| Concern | This module's default (minimum call) | Opt-out (you must type it) |
|---|---|---|
| Authentication | identity-based β WorkspaceSystemAssignedIdentity, storing no credential |
service_data_auth_identity = "None" plus a key or SAS |
| Credential in state | none accepted | supply one; it lands in state and the workspace Key Vault |
| Credential disclosure | presence and count only | no opt-out, by design |
| Default datastore | false β also the only value accepted at creation |
true, on a second apply |
| Storage lifecycle | references only β creates and deletes nothing | none; a datastore cannot own storage |
π Where secure-by-default applies here, and how. Three arguments are required, so there is no empty call in the strict sense β but the one security-relevant default could be flipped and was: the provider's
Nonecaches a storage key behind a workspaceReaderrole; this module'sWorkspaceSystemAssignedIdentitystores nothing. The flip was verified not to remove a legal shape first. The compensations for what cannot be defaulted β closed value sets, the restrictive fact named in each error message, and every unverifiable decision emitted β are enumerated insecure_by_default_applies_here_and_this_is_how.
terraform init -backend=false
terraform validate
terraform fmt -checkPin by tag β ?ref=v1.0.0 β never a branch. This library is plan-only: a human applies from CI.
terraform plan -out=tfplan
terraform show -json tfplan | jq '.planned_values.outputs'
β οΈ Two sequencing notes. To make a datastore the default, apply once withis_default = falseand again withtrue. And confirm the workspace identity'sStorage Blob Data Readerassignment exists before anyone runs a job β a clean apply here says nothing about read access.
β οΈ Never let a retagging sweep reach this resource.tagsis force-new; the sweep will recreate every datastore and break everyazureml://reference mid-flight.
The offline gate is init -backend=false β validate β fmt -check.
What only a root-module terraform console proves: that each validation {} block fires. terraform validate on a
calling configuration does not run root-variable validations. Feed it deliberately bad .tfvars and read the line
numbers, not the message text β Terraform hard-wraps messages, so grepping for a phrase silently matches nothing.
echo 'var.name' | terraform console -var-file=bad.tfvars
β οΈ Suppression is per-variable, so a short error list is not proof a check is missing. And proving the twocredentialrules needs two fixtures pulling in opposite directions: one setting both credentials, and one settingNonewith neither.
What only a real apply exercises: whether the workspace and container exist, whether the workspace identity holds
Storage Blob Data Reader, whether a SAS has expired, whether soft delete blocks identity passthrough, and whether
is_default = true is a create or an update. None of that is visible offline.
id = "/subscriptions/.../resourceGroups/rg-ml-eastus/providers/Microsoft.MachineLearningServices/workspaces/mlw-platform-eastus/dataStores/training_data"
name = "training_data"
datastore_uri_prefix = "azureml://subscriptions/.../resourcegroups/rg-ml-eastus/workspaces/mlw-platform-eastus/datastores/training_data/paths/"
workspace_name = "mlw-platform-eastus"
storage_account_name = "stmlplatformeastus"
storage_account_id = "/subscriptions/.../resourceGroups/rg-ml-eastus/providers/Microsoft.Storage/storageAccounts/stmlplatformeastus"
storage_container_name = "training-data"
authentication_mode = "Identity-based"
service_data_auth_identity_effective = "WorkspaceSystemAssignedIdentity"
uses_identity_based_access = true
stores_no_credential_in_the_workspace_key_vault = true
uses_account_key = false
uses_sas_token = false
supplied_credential_count = 0
credential_supplied_but_identity_selected = false
is_default = false
posture_summary = "Identity-based (WorkspaceSystemAssignedIdentity) / container training-data on stmlplatformeastus"
βΉοΈ
supplied_credential_count = 0alongsideuses_identity_based_access = trueis the shape to aim for: nothing in the workspace Key Vault, nothing in state, nothing to rotate.βΉοΈ Note the URI's lower-case
resourcegroupsagainst the Resource ID'sresourceGroups. Both are correct for their own format, which is exactly why the prefix is emitted rather than hand-built.
| Symptom | Cause | Fix |
|---|---|---|
workspace_id is a bare GUID⦠|
module.mlw.workspace_id was passed. The correctly-named output is the wrong one. |
Pass module.mlw.id. |
| Workspace ID rejected as not a workspace path | A resource group, storage account or child resource ID. | Pass the workspace's own ARM ID; the pattern is anchored at both ends. |
storage_container_id is a FILE SHARE ID⦠|
A /fileServices/default/shares/ path. |
Use the fileshare datastore module; its argument is storage_fileshare_id. |
| Container ID rejected as having no container segment | A bare storage account ID. | Append /blobServices/default/containers/<name>. |
credential may set account_key OR shared_access_signature⦠|
Both supplied. The provider documents them as conflicting. | Choose one β or neither, and use an identity. |
credential.service_data_auth_identity is "None" β¦ and neither account_key nor shared_access_signature was supplied |
None is a commitment to a credential. |
Supply one, or set WorkspaceSystemAssignedIdentity. |
shared_access_signature begins with "?" |
Copied from the portal's SAS token field. | Strip the leading ?; start at sv=. |
shared_access_signature is a full URL |
Copied from the portal's Blob SAS URL field. | Pass only the part after the ?. |
| Identity value rejected as a different vocabulary | SystemAssigned / UserAssigned β the ARM identity {} block's words. |
Use the Workspace-prefixed values. |
| Identity value rejected for case | ARM is case-sensitive here. | Exact camel case. |
Apply fails: is_default cannot be set |
The provider allows true only on update. Uncheckable in a validation. |
Apply with false, then change to true and apply again. |
| Datastore created, but a job cannot read the data | The workspace identity has no Storage Blob Data Reader on the account. No Terraform error, ever. |
Grant it; scope with this module's storage_account_id. |
| As above, and the account is behind a private endpoint | The workspace identity also needs Reader on the private endpoint. |
Grant it on each endpoint in use (blob, file, dfs). |
| Identity passthrough fails on a credential-less blob datastore | Soft delete is enabled on the storage account β a documented constraint. | Disable it, or use a credential. Invisible from here. |
| A datastore that worked stops working, with no plan change | A SAS expired. Nothing in Terraform tracks the expiry. | Rotate the SAS β or move to identity-based access and stop rotating. |
Someone with only workspace Reader obtained the storage key |
Expected, on a credential-based datastore. Microsoft documents it. | Move to identity-based access; review who holds workspace Reader. |
| A retagging sweep destroyed every datastore | tags is force-new on this resource. |
Keep changeable metadata on the workspace or storage account. No data was lost. |
| Notebooks broke after a rename | name is force-new and embedded in every azureml:// URI. |
Recreate under the old name, or update the references. |
A credential key was ignored |
Object-type conversion silently discards extra keys β likely service_data_identity, the sibling's spelling. |
Check service_data_auth_identity_effective. |
terraform destroy β is the data gone? |
No. A datastore is a reference. | Nothing to recover. But every azureml:// reference to it now resolves to nothing. |
azurerm_machine_learning_datastore_blobstorageβ the provider resource.- Data concepts in Azure Machine Learning β the datastore definition, the two authentication mechanisms, and the storage-service support table.
- Set up authentication between Azure ML and other services β where credentials are cached and who can retrieve them.
- Create datastores β worked examples for each authentication mode.
- Use Azure ML studio in a virtual network β the
Storage Blob Data Readerand private-endpointReaderrequirements. SCOPE.mdβ this module's cross-module contract.- Siblings:
terraform-azurerm-machine-learning-workspace(the parent),terraform-azurerm-machine-learning-datastore-fileshare,terraform-azurerm-machine-learning-datastore-datalake-gen2,terraform-azurerm-storage-account,terraform-azurerm-role-assignments,terraform-azurerm-machine-learning-compute-cluster.
π "Infrastructure as Code should be standardized, consistent, and secure."