fix: Rewrite translated field names in all annotate() expressions - #825

annotate(x=F("title")) returns the German value under an active de, but annotate(x=Lower("title")) returns the English one. Every annotation expression that wraps a translated field name, except a bare F() or a Concat() of bare F()s, silently reads the untranslated column: on a row created under en as title_en="title_en", title_de="title_de", Upper(F("title")), Coalesce("title", Value("fallback")) and Case(When(pk__gt=0, then=F("title"))) all return the English value under de. The same gap reaches QuerySet.datetimes(), which Django implements as annotate(datetimefield=Trunc(field_name, kind, output_field=DateTimeField(), tzinfo=tzinfo), plain_field=F(field_name)) and then filters on plain_field__isnull=False: the F() half was rewritten and the Trunc() half was not, so the filter read datetime_de while the returned values read datetime. dates() escapes this only because MultilingualQuerySet overrides it.

MultilingualQuerySet.annotate rewrites a keyword argument only when it passes one of two isinstance checks (modeltranslation/manager.py:531 and modeltranslation/manager.py:533); anything else goes to super().annotate() untouched. _rewrite_f (modeltranslation/manager.py:306) already walks any expression generically through get_source_expressions()/set_source_expressions() and returns what it does not recognise unchanged, which is why _rewrite_filter_or_exclude and update() can already hand it arbitrary values. _rewrite_concat (modeltranslation/manager.py:516) is a hand-rolled special case of that same walk.

The fix replaces the two branches with the call the other rewriting paths already make, kwargs[key] = self._rewrite_f(val), and deletes _rewrite_concat. The generic walk handles Concat on its own, and the existing Concat assertions in test_annotate still pass. One existing shape changes its results: a keyword aggregate is rewritten along with everything else, so annotate(n=Count("title")) now compiles to COUNT("title_de") instead of COUNT("title"). With two rows, one translated and one with title_de=None, de used to give [1, 1] and now gives [1, 0]. That matches filter(title__isnull=False), and there is a test for it.

The widened rewrite has a cost. Rewriting title to title_de swaps the resolved output field from Django's CharField to modeltranslation's TranslationCharField (modeltranslation/fields.py:88 builds TranslationFieldSpecific(TranslationField, baseclass), with CharField as the base class here), and BaseExpression._resolve_output_field takes the first non-None source as the output field and checks that it is an instance of each remaining source's class. A translated field first is fine; a literal first with a translated field after it now raises FieldError: Expression contains mixed types: CharField, TranslationCharField. You must set output_field. Three ordinary shapes go from returning a value to raising: Concat(Value("x"), Lower("title")), Case(When(..., then=Value("a")), When(..., then=Lower("title")), default=Value("d")) and Concat(Upper("title"), Value("-"), Lower("title")). That is the same error Concat(Value("x"), F("title")) already raises today, since F() was already rewritten, and what these three returned before was the untranslated value, so a silently wrong result becomes a loud one. Adding output_field=CharField(), which Django's own Concat documentation already requires for mixed types, makes all three work and return the translated value; test_annotate_literal_before_translated_field pins both halves. Coalesce(Value("x"), F("title")) raises too; it returned the literal before and still does once output_field is set.

Four things this leaves alone. Positional annotate(*args) is untouched, because rewriting Count("title") would change its generated alias from title__count to title_de__count. A Q passed as an aggregate's filter=, and the lookups inside a When() condition, are still not rewritten, since _rewrite_f leaves Q's tuple children alone. QuerySet.alias() is not overridden at all, so alias(x=Lower("title")) still reads the untranslated column, before and after this change. And aggregate() is not overridden either, so aggregate(n=Count("title")) still counts the untranslated column while annotate(n=Count("title")) now counts the current language's; closing that gap wants its own PR and test.

This addresses the other-expressions half of #728, "Support for Subquery and other expressions in annotate()", where last-partizan wrote "you can look at the source code and try to add this feature. I'll gladly review and merge it". The Subquery side of #728 and #702 is a different root cause I have not touched: a plain Subquery(qs.values("title")[:1]) already returns title_de today, because the inner MultilingualQuerySet.values() rewrote the name when the subquery was built, and this change does nothing to it either way, since Query.get_source_expressions() returns [] and _rewrite_f finds nothing to rewrite inside.

One caveat: _rewrite_f mutates expressions in place, so an expression stored at module level and reused across languages keeps the language it was first rewritten for. SHARED = Lower("title") annotated under de and then under en compiles to LOWER("title_de") both times. This is pre-existing for F() and Concat(), but the reach grows to every expression type, and a proper fix wants its own PR and test.

There are four regression tests, one for datetimes() and three for the annotation shapes, each next to the closest existing test. The datetimes() test creates its rows under en on purpose, since setUp activates de and creating under the language you later query would hide the bug. The rewriting-methods list in docs/modeltranslation/usage.rst gains datetimes() and annotate(), with the output_field caveat.

Verified on sqlite only, with Python 3.14.3 and Django 6.0.2 (USE_TZ is False in the test settings, so datetimes() passes tzinfo=None and does no timezone conversion):

$ uv run --no-sync pytest -k "test_annotate_nested_expressions or test_annotate_literal_before_translated_field or test_annotate_keyword_aggregate or test_datetimes_queryset"
4 passed, 197 deselected, 2 warnings in 0.48s
# the same four tests applied on top of unmodified master all fail there:
#   test_annotate_nested_expressions                assert 'title_en' == 'title_de'
#   test_annotate_literal_before_translated_field   assert 'prefix: title_en' == 'prefix: title_de'
#   test_annotate_keyword_aggregate                 assert [1, 0] == [1, 1]
#   test_datetimes_queryset                         At index 0 diff: datetime.datetime(2015, 1, 1, 0, 0) != datetime.datetime(2005, 1, 1, 0, 0)
$ uv run --no-sync pytest
200 passed, 1 skipped, 2 warnings in 1.18s
$ make lint
All checks passed!
42 files already formatted
$ make typecheck
# after two pre-existing UserWarning lines about index names over 30 characters:
Success: no issues found in 38 source files