Python: Bound declarative custom-function matches to one whole call - #9176

Motivation & Context

The declarative expression layer matches its Copilot Studio dialect with patterns anchored to the end of the whole formula, and neither checks that the closing parenthesis belongs to the call being matched:

# _declarative_base.py, DeclarativeWorkflowState._eval_custom_function
re.match(r"(?:Concat|Concatenate)\((.+)\)$", formula.strip())   # + UserMessage, AgentMessage, MessageText

# _state.py, WorkflowState._eval_simple
formula.startswith(f"{func_name}(") and formula.endswith(")")    # + the Not() special case

So any formula that begins with one of these names and ends with ) is captured whole, and everything after the call's own closing paren is swallowed into the argument list. _eval_custom_function runs before PowerFx and eval() returns on any non-None result, so the mis-parse wins and the formula never reaches the engine that would have handled it correctly.

Two symptoms, depending on what the swallowed tail looks like:

For Concat/Concatenate the tail still begins and ends with ", so the literal branch strips the outer quotes and emits the formula's own source as data — silently wrong output, no error. For MessageText the tail is re-evaluated as a PowerFx fragment (="hi") & Upper("there"), raising a spurious compilation error for a formula that is valid PowerFx.

Description & Review Guide

What are the major changes?

• Adds whole_call_args(formula, *names) to _powerfx_functions.py. It scans for the parenthesis that closes the call, tracking nesting depth and string literals (so a ) inside a quoted literal is data, not a terminator), and accepts the match only when that parenthesis is the formula's last character. Argument-less calls return None so PowerFx keeps reporting the arity error, matching the old (.+) behaviour of requiring at least one argument.
• DeclarativeWorkflowState._eval_custom_function uses it for all five names, replacing the four regexes and the now-unused local import re.
• WorkflowState._eval_simple uses it for the CUSTOM_FUNCTIONS dispatch loop and for the Not() special case above it, which had the same flaw. This path is reachable whenever PowerFx itself fails first and execution falls back: WorkflowState().eval('=Concat("a") & Lower("B")') returned the same corrupted a") & Lower("B'.
• 17 tests: unit coverage for the helper (nesting, quoted parens, empty and unbalanced arguments, case sensitivity, name-not-at-start) and behavioural regressions on both evaluators. All of them are engine-free, so they run whether or not dotnet/powerfx is present.

What is the impact of these changes?

A formula that is a single complete call is unaffected — that is the Concat("a", "b") control row above, and the existing suite covers these paths (test_powerfx_yaml_compatibility.py, test_declarative_state_path_safety.py).

One deliberate behaviour change worth a reviewer's attention: Concat(...) mid-expression now falls through to PowerFx rather than being mis-parsed, and standard PowerFx Concat is the table aggregator, so =Concat("a") & Lower("B") raises Invalid number of arguments: received 1, expected 2-3 instead of silently returning a") & Lower("B. A loud, accurate error replaces corrupted data, but it is not ab. The Copilot Studio string alias only ever applied when Concat( started the formula, which is why Concat resolves to two different functions depending on its position — =Lower("B") & Concat("a") already reached real PowerFx and failed this way before this change. Making the alias apply uniformly inside a larger expression needs expression rewriting rather than extent matching, so I have left it out of this PR; Concatenate and & both compose correctly now. Happy to open a follow-up issue if you would like that tracked.

What do you want reviewers to focus on?

Whether falling through to PowerFx is the right call for Concat mid-expression (above), and whether whole_call_args belongs in _powerfx_functions.py — it is the one module both evaluators already import, but it is a parsing helper rather than a PowerFx function, and it is deliberately not registered in CUSTOM_FUNCTIONS.

Two notes on adjacent issues, neither of which this PR touches:

• Python: Declarative concatenation treats quoted expressions as literal text #9072 (concatenation treats quoted expressions as literal text) concerns the argument literal heuristic inside an otherwise correctly-delimited call. This PR fixes the extent of the match, not argument handling — the Concat("a", "b") control row evaluates correctly before and after. The two compose: once the extent is right, Python: Declarative concatenation treats quoted expressions as literal text #9072's fix governs the arguments. That author mentioned they have a fix in progress, so I have kept this change clear of the argument-parsing branch to avoid a conflict.
• Python: [Bug]: UserMessage/AgentMessage return different shapes from WorkflowState and DeclarativeWorkflowState #9041 (UserMessage/AgentMessage return different shapes from the two state classes) is why MessageText(...) returns '' for a message list in some paths. That is independent of this change — it reproduces identically on the untouched whole-call and nested paths.

Related Issue

Fixes #9173

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change.

uv run poe --directory packages/declarative test      1365 passed (1347 before, +18)
uv run poe --directory packages/declarative syntax    ruff format + check: All checks passed
uv run poe --directory packages/declarative pyright   0 errors, 0 warnings, 0 informations
uv run poe --directory packages/declarative mypy      12 errors in 6 files, identical on an
                                                      unmodified tree - all pre-existing

>       assert state._eval_simple(formula) == formula
E       assert 'a") & Lower("B' == 'Concat("a") & Lower("B")'