Add option to disable automatic last message loading by sunlei · Pull Request #3731 · tdlib/td

sunlei · 2026-08-24T11:25:44Z

Summary

This PR adds a writable Boolean option named disable_automatic_last_message_loading.

When use_message_database is false and this option is true, TDLib no longer automatically requests message history through load_last_dialog_message to populate a chat's last message.

The option defaults to false, so the existing behavior remains unchanged for current clients.

Context: #62, #1605, #2286, #2893

Background

TDLib's local database grows over time, and the message database currently has no size limit. This has been discussed in #62 and #1605.

In #2893, disabling all databases through setTdlibParameters is suggested for applications that don't need them, so that db.sqlite is no longer used.

However, when the message database is disabled, TDLib can't restore a dialog's last message from the local database. load_last_dialog_message currently calls get_history_impl, which may eventually send a remote messages.getHistory request.

#2286 documents this behavior: when the last message of a chat becomes unknown, TDLib automatically calls getChatHistory to repair it. A large number of such requests may result in FLOOD_WAIT errors.

Automatic last-message loading can be triggered by multiple internal paths, including:

initializing or loading a dialog;
updating a dialog position;
completing a channel difference;
deleting the current last message.

Disabling the databases can therefore cause a history-request fan-out that the application can't control. For accounts with many dialogs, these automatic requests may result in FLOOD_WAIT errors.

Some applications intentionally disable the message database and can handle chats whose last_message is temporarily unknown. For these applications, automatically loading history isn't required.

Changes

This PR adds the following check to the common MessagesManager::load_last_dialog_message entry point:

if (!G()->use_message_database() &&
    td_->option_manager_->get_option_boolean("disable_automatic_last_message_loading")) {
  return;
}

It also registers disable_automatic_last_message_loading as a writable Boolean option in OptionManager.

Placing the check in the common entry point covers all current automatic loading paths that use load_last_dialog_message, without changing each call site separately.

Option semantics

disable_automatic_last_message_loading:

Type: Boolean
Writable: yes
Default: false
Effective only when use_message_database == false

When enabled, TDLib doesn't automatically load message history to populate a chat's last message. The application must be able to handle chats with an unknown last message.

The option has no effect when the message database is enabled.

Unaffected behavior

This change doesn't prevent:

explicit application calls to getChatHistory;
pagination and continuation of explicit history requests;
getDifference or getChannelDifference requests;
new-message updates from updating a chat's last message;
other TDLib background requests.

The change only gates automatic history loads made through load_last_dialog_message.

Why this is opt-in

Unconditionally skipping automatic last-message loading whenever the message database is disabled would change the behavior of all existing database-less clients.

Some clients may rely on TDLib to populate chat.last_message automatically. A disabled-by-default option preserves backward compatibility while allowing applications that don't need this behavior to opt out explicitly.

Why not use `ignore_background_updates`

ignore_background_updates changes offline update recovery and update-state persistence, so its scope is much broader than last-message loading.

The new option only controls automatic last-message loading and doesn't modify update-difference code paths.

Usage

The setOption request needs to be sent to TDLib before the setTdlibParameters request. The client doesn't need to wait for the setOption response before sending the initialization parameters.

First send:

{
  "@type": "setOption",
  "name": "disable_automatic_last_message_loading",
  "value": {
    "@type": "optionValueBoolean",
    "value": true
  }
}

Then initialize a database-less client with the relevant fields set to:

{
  "@type": "setTdlibParameters",
  "use_file_database": false,
  "use_chat_info_database": false,
  "use_message_database": false
}

Trade-offs

When the option is enabled:

last_message may remain unknown for some chats until TDLib learns it from a subsequent update, a dialog response, or an explicit application request;
less local message context is available without the message database;
the application needs to accept the existing functional limitations of running without a local message cache.

These changes apply only to clients that explicitly enable the option.

Non-goals

This PR doesn't address:

size limits or retention while the message database is enabled;
SQLite cleanup, compaction, or vacuuming;
other background requests such as getPeerDialogs, getPeerSettings, or getBirthdays;
full semantic equivalence between database-less mode and mode with all databases enabled.

Validation

tdcore builds successfully.
run_all_tests passes.
The clang-format check passes.
git diff --check passes.

100% tests passed, 1 test out of 1
Total Test time: 417.54 sec

- Skip automatic last-message history requests without message database. - Keep the current behavior by default.

levlam · 2026-08-24T16:50:21Z

The last message is required for all chats in the user chat list to maintain the correct position of the chat in the list. Any such change will break app UX and therefore can't be accepted to the upstream, but you are free to add it to your fork of TDLib.

sunlei · 2026-08-24T16:55:32Z

@levlam Thanks, I understand the concern now.

The option being disabled by default preserves behavior for existing clients, but I agree that enabling it can leave the last message unknown and therefore keep the chat position stale. I also noticed that the TDLib API explicitly allows messages to be added without a corresponding updateNewMessage while the last message is unknown, so leaving this state unrepaired isn't a safe general-purpose mode.

The problem I am trying to solve is the per-dialog getChatHistory fan-out in database-less mode, rather than removing the last-message invariant itself.

Would an implementation that preserves automatic last-message repair, but batches or coalesces repairs for multiple dialogs, be acceptable upstream?

levlam · 2026-08-24T18:49:11Z

There is no way to "batch" the requests. The last message hardly ever needs to be repaired, therefore regular users have no issues. Bots don't need the last chat message at all. This behavior isn't optional, it is something that must be done always.

omd2you-a11y · 2026-08-25T03:10:27Z

Hi

sunlei · 2026-08-25T06:14:24Z

@levlam I agree that the last-message invariant must not be optional. If the proposed option can violate that invariant, then this PR is not the correct implementation.

However, the cold-start behavior we observed with a real user account still seems unresolved:

With all three databases disabled, one restart produced 192 FLOOD_WAIT log entries, including 149 messages.getHistory queries with distinct internal query IDs.
After re-enabling all three databases, the first restart with an empty database produced flood log entries for 1,142 distinct query IDs, including:
- 354 messages.getHistory
- 306 messages.getPeerDialogs
- 374 channels.getFullChannel
- 108 stories.getPeerStories

This suggests that the issue is not merely whether last-message loading can be skipped. When TDLib starts without reusable local state, it sends a concentrated fan-out of automatic RPCs. Disabling all databases makes every restart such a cold start.

Could you please clarify the intended support contract?

For a user account, is setting all of the following to false a supported configuration?
- use_file_database
- use_chat_info_database
- use_message_database
If it is supported, how is TDLib expected to restore the mandatory last message and other required state without producing this request fan-out and repeatedly triggering FLOOD_WAIT after restarts?
If this configuration is actually intended only for bot accounts, should that restriction be documented or enforced when setting the parameters?

I accept that this PR is not the correct solution if it breaks the last-message invariant. But the cold-start request amplification still appears to need an upstream solution, for example by persisting bounded essential state when the full message database is disabled, restoring non-immediate state lazily, or coordinating automatic background requests so that they are not fanned out at startup.

The goal is not to make TDLib's state invariants optional. It is to restore them in a way that allows supported database-less or empty-database startup without repeatedly triggering FLOOD_WAIT.

levlam · 2026-08-25T11:59:26Z

When TDLib starts without reusable local state, it sends a concentrated fan-out of automatic RPCs.

It doesn't. TDLib shouldn't exceed any flood limits by itself. Could you send a TDLib log with verbosity level 4 to https://t.me/tdlib_bot, so I can see the cause of the requests?

sunlei · 2026-08-25T16:00:49Z

@levlam I sent the requested verbosity level 4 log to @tdlib_bot.

The reproduction used TDLib 1.8.66 (a9966eb3704a3351568c28013fed67d797c17828) with a user account and all three databases disabled:

use_file_database = false
use_chat_info_database = false
use_message_database = false

The application didn't call getChatHistory and performed no application-level retry or automatic restart.

The log shows the following sequence:

MessagesManager initiated history loading for 486 distinct chats with:

Get history from the end of chat ... from server from fix_new_dialog 15

These operations created messages.getHistory requests (0x4423e6c5).
The Telegram server returned error 420 FLOOD_WAIT_n, and NetQueryDelayer delayed and retried the requests.
After their accumulated delay/timeout limits were exhausted, TDLib generated error 429 Too Many Requests for 111 distinct query IDs.

A correction to my earlier comment: the server didn't directly return error 429. It returned error 420 FLOOD_WAIT_n; the final 429 results were generated by TDLib after the delayed queries exhausted their total timeout.

The minimized sanitized bundle sent to @tdlib_bot is 69,415 bytes. It contains the startup evidence, all 486 fix_new_dialog repair triggers, the recorded lifecycles of the 111 queries that ended with TDLib error 429, and one representative end-to-end query trace. I retained the full sanitized verbosity-4 log in case more context is needed.

The related issues describe different parts of the same causal chain rather than duplicate reports:

Ability to optimize Local Database #62 and Local cache configuration? #1605 document that the message database can grow without a size limit.
In db.sqlite keeps growing forever.. up to 18 Gig! #2893, disabling all databases was suggested as a way to avoid that unbounded database.
Error 420 Flood Wait Failed #2286 documents that TDLib automatically calls getChatHistory when the last message becomes unknown and that these requests can encounter FLOOD_WAIT.
This reproduction shows the interaction: disabling all three databases caused one startup to schedule last-message repair for 486 chats.

This doesn't change your point that the last-message invariant is mandatory, and it doesn't make the current opt-out implementation acceptable upstream. It demonstrates the underlying database-less cold-start fan-out that motivated the PR.

Could you check whether all 486 repairs being started from fix_new_dialog 15 during one database-less startup is expected, or whether some state or scheduling mechanism is supposed to prevent this request concentration?

Add option to disable automatic last message loading

49ead42

- Skip automatic last-message history requests without message database. - Keep the current behavior by default.

Forrie083 approved these changes Aug 28, 2026

View reviewed changes

Navigation Menu

Add option to disable automatic last message loading - #3731

Add option to disable automatic last message loading#3731
sunlei wants to merge 1 commit into
tdlib:masterfrom
sunlei:sunlei/disable-automatic-last-message-loading

sunlei commented Aug 24, 2026

Uh oh!

levlam commented Aug 24, 2026

Uh oh!

sunlei commented Aug 24, 2026

Uh oh!

levlam commented Aug 24, 2026

Uh oh!

omd2you-a11y commented Aug 25, 2026

Uh oh!

sunlei commented Aug 25, 2026

Uh oh!

levlam commented Aug 25, 2026

Uh oh!

sunlei commented Aug 25, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sunlei commented Aug 24, 2026

Summary

Background

Changes

Option semantics

Unaffected behavior

Why this is opt-in

Why not use ignore_background_updates

Usage

Trade-offs

Non-goals

Validation

Uh oh!

levlam commented Aug 24, 2026

Uh oh!

sunlei commented Aug 24, 2026

Uh oh!

levlam commented Aug 24, 2026

Uh oh!

omd2you-a11y commented Aug 25, 2026

Uh oh!

sunlei commented Aug 25, 2026

Uh oh!

levlam commented Aug 25, 2026

Uh oh!

sunlei commented Aug 25, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Why not use `ignore_background_updates`