☁️ Azure Data Factory Dataset — PostgreSQL Terraform Module

A typed Data Factory dataset describing a table in a PostgreSQL database reached through a linked service, targeting hashicorp/azurerm ~> 4.0.

🧩 Overview

🐘 Creates one azurerm_data_factory_dataset_postgresql — a named description of a table in a PostgreSQL database that pipelines read from and write to.
👯 Is one of three schema-identical resources — with the SQL Server Table and MySQL datasets — the only true clone cluster in the Data Factory dataset family.
✅ Its requires-import error names its own resource type correctly — worth stating, because two of its siblings get that wrong.
🔌 Names only a table. The server and the database are chosen by the linked service.
🧾 Has no schema or database argument — unlike the Snowflake dataset, which carries schema_name.

💡 Why it matters: the family has refuted the clone hypothesis three times, and here it finally holds at the argument surface. It still does not hold in the provider code — the schema-identical SQL Server Table resource raises a requires-import error naming a type that does not exist.

❤️ Support this project

If this module saved you time:

⭐ Star the repository — it is the cheapest signal that this work is worth continuing.
💼 Connect on LinkedIn: linkedin.com/in/microsoftexpert
☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

🗺️ Where this fits in the family

flowchart TB
  RG["terraform-azurerm-resource-group"]
  ADF["terraform-azurerm-data-factory"]
  LS["a PostgreSQL linked service, which chooses the server AND the database"]
  THIS["terraform-azurerm-data-factory-dataset-postgresql"]
  SQLS["terraform-azurerm-data-factory-dataset-sql-server-table"]
  MYSQLDS["terraform-azurerm-data-factory-dataset-mysql"]
  AZSQL["terraform-azurerm-data-factory-dataset-azure-sql-table"]
  PIPE["a Data Factory pipeline"]
  TBL["a table in a PostgreSQL schema"]

  RG -->|"name"| ADF
  ADF -->|"id"| THIS
  ADF -->|"id"| SQLS
  ADF -->|"id"| MYSQLDS
  ADF -->|"id"| AZSQL
  LS -->|"linked_service_name, a bare NAME"| THIS
  LS -->|"linked_service_id, a full Resource ID"| AZSQL
  THIS -->|"schema-identical to these two"| SQLS
  THIS -->|"schema-identical to these two"| MYSQLDS
  THIS -->|"name, referenced by"| PIPE
  PIPE -->|"reads only at run time, never here"| TBL

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef keystone fill:#004578,stroke:#00243d,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class ADF keystone
  class RG,LS,SQLS,MYSQLDS,AZSQL,PIPE,TBL sibling

The linked service node carries the fact most easily missed: it chooses the server and the database. This dataset names only a table — and on PostgreSQL, not even the schema.

🧬 What this module builds

flowchart TB
  subgraph INPUTS["Inputs"]
    NAME["name (force-new)"]
    ADFID["data_factory_id (force-new)"]
    LSNAME["linked_service_name, a bare NAME"]
    TBL["table_name, optional and unqualified"]
    COLS["schema_column list"]
    META["parameters, additional_properties, annotations, description, folder"]
  end

  THIS["the dataset resource -- this shape is shared by the schema-identical trio"]

  subgraph OUTPUTS["Outputs"]
    OID["id and name"]
    OTBL["table_name and has_table_name"]
    ONONE["describes_no_specific_table"]
    OQUAL["table_name_looks_qualified"]
    OUNTYPED["untyped_schema_column_names"]
    OGUARD["the import-guard fact, which is the ONE output that differs across the trio"]
  end

  NAME --> THIS
  ADFID --> THIS
  LSNAME --> THIS
  TBL --> THIS
  COLS --> THIS
  META --> THIS

  THIS --> OID
  TBL --> OTBL
  TBL --> ONONE
  META --> ONONE
  TBL --> OQUAL
  COLS --> OUNTYPED
  THIS --> OGUARD

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class NAME,ADFID,LSNAME,TBL,COLS,META,OID,OTBL,ONONE,OQUAL,OUNTYPED,OGUARD sibling

ℹ️ This shape diagram is shared with the SQL Server Table and MySQL modules, because all three resources are schema-identical. Drawing two different pictures would invent a distinction a reader would then have to go and verify.

Resource	Count	Notes
`azurerm_data_factory_dataset_postgresql.this`	1	The keystone. A real ARM child of the factory.
`schema_column`	`dynamic`, 0..n	Optional column definitions.
`timeouts`	`dynamic`, 0..1	All four keys exist.

✅ Provider / Versions

Requirement	Value
Terraform	`>= 1.12.0`
`hashicorp/azurerm`	`~> 4.0`
Provider block	None. The caller configures the provider, including the mandatory `features {}` block.

Schema notes that bite — verified against the live provider source, not inferred from the schema:

🔴 This resource, the SQL Server Table dataset and the MySQL dataset are schema-identical — same arguments, blocks, types, optionality and validators, with byte-identical table_name definitions (TypeString, Optional, StringIsNotEmpty).
🔴 BYTE-IDENTICAL SCHEMAS DO NOT IMPLY IDENTICAL PROVIDER CODE. This resource's requires-import error names azurerm_data_factory_dataset_postgresql, correctly. The SQL Server Table resource's names azurerm_data_factory_dataset_sql_server — a type that does not exist. The JSON dataset's names the delimited-text type. Three resources, three levels of correctness in the same one-line call.
⚠️ There is no database or schema argument. The linked service chooses the database; the provider sends table_name unchanged and offers nowhere else to put a qualifier. The Snowflake dataset does carry a separate schema_name.
⚠️ table_name is optional, and a dataset with neither a table nor parameters applies cleanly and resolves to nothing.
⚠️ linked_service_name is a bare NAME. Across the twelve-resource family only azurerm_data_factory_dataset_azure_sql_table takes an ID.
⚠️ Only name and data_factory_id force replacement.
⚠️ Timeouts default to 30m / 5m / 30m / 30m, all four keys present.

🔑 Required Azure RBAC Roles / Permissions

Permission	Scope	Why
`Microsoft.DataFactory/factories/datasets/write`	the Data Factory	Create and update.
`Microsoft.DataFactory/factories/datasets/read`	the Data Factory	Refresh and plan.
`Microsoft.DataFactory/factories/datasets/delete`	the Data Factory	Destroy — see the warning below.
`Data Factory Contributor`	the Data Factory	The built-in role containing all three.

🔒 No permission on the PostgreSQL server is required or used, and none is available to grant — PostgreSQL authentication is not an Azure RBAC concern. Access belongs to the linked service, so whoever can write datasets can describe any table in whatever database that linked service points at.

⚠️ Delete is the operation to review, not create. Removing a dataset breaks every pipeline referencing it by name.

Azure Prerequisites

An existing Data Factory, and its Resource ID.
A PostgreSQL linked service already configured in that factory, and its name. It chooses the server and the database. Nothing here verifies it.
A table for the dataset to describe — or parameters. Neither is enforced.
A dataset name meeting Microsoft's Data Factory naming rules. The provider checks only non-emptiness.
The Microsoft.DataFactory resource provider registered in the subscription.

📁 Module Structure

terraform-azurerm-data-factory-dataset-postgresql/
├── providers.tf    # required_version + the pinned azurerm; no provider block
├── variables.tf    # 11 typed inputs, 12 validations
├── main.tf         # the keystone, its locals, and two dynamic blocks
├── outputs.tf      # 48 outputs; id first
├── README.md       # this file
├── SCOPE.md        # the cross-module contract
├── LICENSE         # MIT
└── .gitignore

⚙️ Quick Start

provider "azurerm" {
  features {}
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name

  table_name = "orders"
}

ℹ️ No database is named here — the linked service chose it.

🔌 Cross-Module Contract

Consumes

Input	Type	Typical source
`data_factory_id`	`string`	`terraform-azurerm-data-factory` → `id`
`linked_service_name`	`string`	a PostgreSQL linked service's name
`table_name`	`string`, optional	the caller

Emits (selected — 48 in total)

Output	Description
`id`	The dataset's Resource ID.
`table_name`, `has_table_name`	What it points at.
`describes_no_specific_table`	No table and no parameters.
`this_resource_has_no_separate_schema_or_database_argument`	Constant. The linked service chooses the database.
`the_import_guard_names_this_resource_correctly`	Constant. Two siblings get this wrong.
`this_resource_is_schema_identical_to_the_sql_server_table_and_mysql_datasets`	Constant.

📚 Example Library

1 · The minimum call

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
}

⚠️ Legal, and it describes nothing a pipeline can resolve — no table, no parameters. describes_no_specific_table is true.

2 · Naming a table

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name

  table_name = "orders"
}

💡 Unqualified is the normal case here — the database came from the linked service, and this resource has no argument for one.

3 · The database is not yours to choose here

# The SAME dataset definition reads from a different database purely by
# pointing at a different linked service.
module "orders_prod" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders_prod"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_prod_linked_service_name
  table_name          = "orders"
}

module "orders_staging" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders_staging"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_staging_linked_service_name
  table_name          = "orders"
}

⚠️ Changing linked_service_name on an existing dataset is an in-place update, so a dataset can be moved between production and staging databases without a destroy appearing in the plan.

4 · Passing a Resource ID is refused

# ❌ NOT ACCEPTED — this resource takes a NAME
module "wrong_parent_reference" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = "/subscriptions/.../linkedservices/ls_mysql" # an ID
}

ℹ️ Only azurerm_data_factory_dataset_azure_sql_table takes an ID, out of the whole twelve-resource family.

5 · Guarding against a dataset that resolves to nothing

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = var.table_name # may be null
}

check "dataset_points_somewhere" {
  assert {
    condition     = !module.orders.describes_no_specific_table
    error_message = "ds_orders names no table and exposes no parameters; no pipeline can resolve it."
  }
}

6 · A parameterised dataset

module "any_table" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_any_table"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name

  parameters = {
    tableName = ""
  }
}

💡 With parameters set, describes_no_specific_table is false even with no table.

7 · A typed column schema

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = "orders"

  schema_column = [
    { name = "order_id", type = "Int64", description = "Primary key." },
    { name = "placed_at", type = "DateTimeOffset" },
    { name = "notes" }, # untyped -- legal
  ]
}

check "every_column_is_typed" {
  assert {
    condition     = length(module.orders.untyped_schema_column_names) == 0
    error_message = "Untyped columns: ${join(", ", module.orders.untyped_schema_column_names)}"
  }
}

🔒 The fifteen types are case-sensitive, and are Data Factory's types rather than PostgreSQL's — Int64, not BIGINT.

8 · Lowercase column types are refused

# ❌ NOT ACCEPTED — the type set is case-sensitive
module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name

  schema_column = [{ name = "order_id", type = "int64" }] # wants "Int64"
}

9 · Many tables from one set

locals {
  tables = ["orders", "customers", "ledger"]
}

module "tables" {
  for_each = toset(local.tables)

  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_${each.key}"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = each.key

  folder = "bronze/postgresql"
}

check "no_table_resolves_to_nothing" {
  assert {
    condition     = alltrue([for m in module.tables : !m.describes_no_specific_table])
    error_message = "At least one dataset names no table."
  }
}

💡 The provider takes no lock on the Data Factory for this resource, so these are created concurrently and safely.

10 · Annotations, description and an authoring folder

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = "orders"

  description = "Order header rows, read hourly by the curated pipeline."
  folder      = "bronze/postgresql"
  annotations = ["bronze", "hourly"]
}

ℹ️ annotations is a list of strings, unrelated to Azure resource tags — which this resource does not support. Tag the Data Factory instead.

11 · The `additional_properties` escape hatch

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = "orders"

  additional_properties = {
    "structure" = "[]"
  }
}

⚠️ Merged as top-level keys alongside the managed fields. Nothing validates them.

12 · Importing — and why this module says its guard is correct

module "existing" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_existing"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = "orders"
}

output "import_with" {
  value = module.existing.import_address
}

✅ This resource's requires-import error names azurerm_data_factory_dataset_postgresql — correct. The schema-identical SQL Server Table resource names a type that does not exist, and the JSON dataset names a different real type. the_import_guard_names_this_resource_correctly records the contrast, so the divergence is visible from either module.

13 · Custom timeouts

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.postgresql_linked_service_name
  table_name          = "orders"

  timeouts = {
    create = "45m"
    read   = "10m"
    update = "45m"
    delete = "45m"
  }
}

ℹ️ Defaults are 30m / 5m / 30m / 30m. Terraform silently discards an object key the type does not declare.

14 · 🏗️ End-to-end composition

provider "azurerm" {
  features {}
}

module "rg" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"

  name     = "rg-analytics-eastus"
  location = "eastus"
}

module "adf" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory.git?ref=v1.0.0"

  name                = "adf-analytics-eastus"
  resource_group_name = module.rg.name
  location            = module.rg.location

  identity = {
    type = "SystemAssigned"
  }
}

# No linked-service module exists in this suite yet, so the linked service is
# declared directly. Its connection string chooses the SERVER and the DATABASE;
# provision the credential out of band and reference it, never inline.
resource "azurerm_data_factory_linked_service_postgresql" "app" {
  name            = "ls_postgresql_app"
  data_factory_id = module.adf.id

  connection_string = var.postgresql_connection_string # sensitive; supplied out of band
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-postgresql.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = module.adf.id
  linked_service_name = azurerm_data_factory_linked_service_postgresql.app.name

  table_name = "orders"
  folder     = "bronze/postgresql"

  schema_column = [
    { name = "order_id", type = "Int64" },
    { name = "placed_at", type = "DateTimeOffset" },
  ]

  annotations = ["bronze", "hourly"]
}

check "wiring_is_consistent" {
  assert {
    condition     = !module.orders.describes_no_specific_table
    error_message = "The dataset resolves to nothing."
  }
}

output "dataset_id" {
  value = module.orders.id
}

output "reads_from" {
  value = module.orders.table_name
}

🔒 The PostgreSQL connection string carries the credential and belongs to the linked service. Keep it in a secret store and pass a reference; this module accepts no secret and emits none. Note that sensitive = true redacts plan output and does not encrypt state — the protection that matters is an encrypted, access-controlled backend.

📥 Inputs

Required: name, data_factory_id, linked_service_name. What it points at: table_name, parameters. Shape: schema_column, additional_properties. Metadata: description, folder, annotations. Tail: timeouts. There is no tags variable — the resource supports none.

Full input schemas

Name	Type	Default	Notes
`name`	`string`	—	Force-new. Non-empty only; a leading `/` is refused.
`data_factory_id`	`string`	—	Force-new. Anchored Resource-ID validator.
`linked_service_name`	`string`	—	A bare name. A Resource ID is refused. It chooses the server and the database.
`table_name`	`string`	`null`	Optional. No database or schema qualifier belongs here.
`schema_column`	`list(object({ name, type, description }))`	`[]`	Closed, case-sensitive set of fifteen types.
`parameters`	`map(string)`	`{}`	Supplied per pipeline run.
`additional_properties`	`map(string)`	`{}`	Unvalidated top-level keys.
`annotations`	`list(string)`	`[]`	Not tags.
`description`	`string`	`null`	Empty string refused; omission accepted.
`folder`	`string`	`null`	The authoring tree.
`timeouts`	`object({ create, read, update, delete })`	`null`	30m / 5m / 30m / 30m.

🧾 Outputs

Output	Description	Notes
`id`	The dataset's Resource ID.	First, by convention.
`name`, `data_factory_id`, `data_factory_name`	Identity and parent.	Name parsed from the end of the ID.
`resource_group_name`, `subscription_id`	Where the factory lives.
`linked_service_name`	The linked service, as supplied.	Chooses the server and database.
`table_name`, `has_table_name`	What it points at.
`describes_no_specific_table`	No table and no parameters.	Assert on this.
`table_name_looks_qualified`	Contains a dot.	Reported, never enforced.
`this_resource_has_no_separate_schema_or_database_argument`	Constant.
`parameters`, `parameter_count`	The run-time inputs.
`schema_column_count`, `has_schema_columns`, `schema_column_names`, `untyped_schema_column_names`	The column schema.
`the_schema_column_type_set_is_closed_and_case_sensitive`	Constant.	Fifteen values.
`the_schema_column_block_is_identical_on_ten_of_the_twelve_datasets`	Constant. Identical on ten of twelve; `binary` has no block and `snowflake`'s differs.	Eleven of twelve.
`uses_additional_properties`, `additional_property_count`	The escape hatch.
`annotations`, `annotation_count`, `has_annotations`	A list, not tags.
`description`, `has_description`, `folder`, `has_folder`	Metadata.
`force_new_fields`, `fields_that_can_change_after_creation`, `fields_azure_returns_on_read`	The change surface.
`import_address`	The Resource ID to import.
`the_import_guard_names_this_resource_correctly`	Constant.	Two siblings get this wrong.
`this_resource_is_schema_identical_to_the_sql_server_table_and_mysql_datasets`	Constant.	The only true clone cluster.
`this_resource_takes_a_linked_service_NAME_not_an_ID`	Constant.	The family splits 11-to-1.
`the_linked_service_is_not_verified_to_exist`	Constant.
`this_dataset_moves_no_data_by_itself`	Constant.
`the_module_cannot_see_which_pipelines_use_this`	Constant.	Read before destroying.
`the_module_cannot_verify_the_table_exists`, `no_credential_is_configured_here`	Constants.
`destroying_the_factory_destroys_this_dataset_too`, `destroying_this_does_not_touch_the_table`	Constants.
`lifecycle_prevent_destroy_is_not_available_to_a_module_caller`	Constant.
`this_is_a_real_azure_resource_not_a_composite`, `the_provider_takes_no_lock_on_the_data_factory`	Constants.
`this_resource_supports_no_azure_resource_tags`	Constant.	Why there is no `tags` variable.
`no_secret_is_accepted_or_emitted_by_this_module`	Constant.

No output is sensitive, and none can be.

🧠 Architecture Notes

The clone cluster is real, and it stops at the schema. This resource, the SQL Server Table dataset and the MySQL dataset declare exactly the same arguments and blocks, with byte-identical table_name definitions. That is the first time in this family the resemblance has survived inspection — the parent reference, the location rules, the compression enums and the schema_column block have each broken it before. And the members still differ in their provider code: this one's requires-import guard is correct, the SQL Server Table one names a type that does not exist, and the JSON dataset's names a different real type. The offline harness proves the agreement by running the same fixtures through both modules and asserting the results match, then asserts the divergence separately.

The linked service chooses the database, and that is the fact most easily missed. This dataset names a table and nothing else. Two datasets with identical definitions read from different databases purely because they point at different linked services — and since linked_service_name is not force-new, moving one between production and staging is an in-place attribute change that no destroy-scan will surface.

There is no schema argument at all, unlike the Snowflake dataset. table_name_looks_qualified reports whether a dot is present and does not enforce anything: a legal PostgreSQL table name may contain one, and an unqualified name is the normal case.

Only identity forces replacement. lifecycle is not valid inside a module block, so a caller cannot add prevent_destroy; a CanNotDelete lock prevents deletion but not the replacement that editing name would cause.

The module cannot see downstream. Pipelines reference a dataset by name and nothing points back.

🧱 Design Principles

Concern	This module's default	Opt-out
Secrets	None accepted, none emitted. The connection string lives on the linked service.	Not available — by design.
Credentials	Live on the linked service, never here.	—
PostgreSQL access	No permission required or used, and none is grantable through Azure RBAC.	—
A Resource ID passed as `linked_service_name`	Refused — a leading `/` is the probable mistake.	None; pass a name.
A dataset that resolves to nothing	Reported via `describes_no_specific_table`.	Ignore the output.
A qualified table name	Reported, never enforced.	Ignore the output.
Column types	Enforced against the provider's case-sensitive fifteen.	None — the set is closed.
`tags`	Not offered — the resource supports none.	Tag the factory.

🔒 There is no risky toggle to close here: the resource holds no credential and touches no data. The exposures are the delete and the silent database switch that a linked-service change performs — the module documents both.

🚀 Runbook

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module with ?ref=v1.0.0 — never a branch. This library is plan-only: a human applies from CI.

🧪 Testing

terraform plan and the module's own validation {} blocks cover, offline and without credentials — note it is plan and not terraform validate, which through a module call evaluates no variable values and reports success:

the anchored Resource-ID shape of data_factory_id, and the refusal of an ID passed as linked_service_name;
the case-sensitive schema_column.type set;
the empty-string rules on table_name, description, folder and each annotation;
every derived flag — describes_no_specific_table, table_name_looks_qualified, untyped_schema_column_names — each with one fixture per branch through terraform console.

The clone claim is executed, not asserted. The harness runs every negative, positive and derived-value fixture through both this module and the SQL Server Table module and asserts they agree — 80 module-runs — then asserts separately that the import-guard outputs differ, which is the one place they must not match.

Only a real plan or apply can tell you whether the linked service exists, whether the table exists, or whether a pipeline still depends on the dataset.

💬 Example Output

dataset_id = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-analytics-eastus/providers/Microsoft.DataFactory/factories/adf-analytics-eastus/datasets/ds_orders"
reads_from = "orders"

linked_service_name         = "ls_postgresql_app"
has_table_name              = true
describes_no_specific_table = false
table_name_looks_qualified  = false
schema_column_names         = ["order_id", "placed_at"]
untyped_schema_column_names = []
force_new_fields            = ["name", "data_factory_id"]

🔍 Troubleshooting

Symptom	Cause	Fix
`linked_service_name must be a bare linked service NAME, not a Resource ID.`	A Resource ID was passed.	Pass the linked service's name; only the Azure SQL Table dataset takes an ID.
`every schema_column type must be one of Byte, Byte[], ...` on `"int64"`	Case-sensitive, and these are Data Factory's types rather than PostgreSQL's.	Use `"Int64"`, not `"bigint"`.
`table_name may be omitted, but must not be set to an empty`	A blank string was passed.	Omit the argument, or give it a value.
A dataset applies but no pipeline can use it	No table and no parameters.	Check `describes_no_specific_table`.
A configuration with a `database` or `schema` argument will not plan	This resource has neither.	The linked service chooses the database.
The dataset suddenly reads from a different database	`linked_service_name` changed — an in-place update.	Review in-place dataset changes, not only replacements.
A pipeline breaks after a clean destroy	Pipelines reference datasets by name; nothing points back.	Check consumers before destroying; consider a `CanNotDelete` lock.
A `timeouts` key seems to have no effect	Terraform silently discards an undeclared object key.	Compare against `object({ create, read, update, delete })`.

🔗 Related Docs

azurerm_data_factory_dataset_postgresql
azurerm_data_factory_dataset_sql_server_table — schema-identical
azurerm_data_factory_dataset_azure_sql_table — takes an ID, not a name
Azure Data Factory datasets
PostgreSQL connector
SCOPE.md — this module's cross-module contract
Sibling modules: terraform-azurerm-data-factory, terraform-azurerm-data-factory-dataset-sql-server-table, terraform-azurerm-data-factory-dataset-azure-sql-table

💙 "Infrastructure as Code should be standardized, consistent, and secure."

Navigation Menu

Repository files navigation

☁️ Azure Data Factory Dataset — PostgreSQL Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Latest commit

History

Folders and files

Repository files navigation

☁️ Azure Data Factory Dataset — PostgreSQL Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required Azure RBAC Roles / Permissions

Azure Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages