feat: load remote ignore patterns (fixes #9352) by calmh · Pull Request #10669 · syncthing/syncthing

calmh · 2026-04-25T10:34:14Z

This adds support for loading ignore files remotely on initial folder setup. That is, it's fine to create a folder with an .stignore that says #include some-other-file, as long as some-other-file is available from a peer device.

This is accomplished by adding a new filesystem shim RemoteFilesystem which serves Stat()/Lstat() by returning the global index entry from the database, and serves Open() calls by reading the file from a remote peer and buffering it in-memory. Obviously this is inefficient for general usage, but it's perfectly fine for ignore files.

The ignore loader then gets a layered view of the local filesystem + a remote filesystem that serves files that don't exist locally, via the new LayeredFilesystem abstraction.

There's a fair amount of new code in the fs package, but it's quite conceptually simple, a lot of it just being wrapping and not-implemented stuff.

The folder, on initial setup, will flip to errored when the ignore file is unavailable both locally and remotely (because we haven't received the index yet). Then on a retry, it'll load the ignores and continue. This is a wrinkle, but I think it's fine for now...

tomasz1986 · 2026-04-25T14:05:43Z

Just a quick question, but what is going to happen when the #include file is located outside of the Syncthing folder (i.e. using .. and relative paths)? I don't do that but I've seen some users do. Is the folder just going to end up getting stopped (as is the case right now)?

calmh · 2026-04-25T14:11:24Z

Yup that gets rejected when it tries to load it and the folder remains stopped.

This adds support for loading ignore files remotely on initial folder setup. That is, it's fine to create a folder with an .stignore that says `#include some-other-file`, as long as some-other-file is available from a peer device. This is accomplished by adding a new filesystem shim `RemoteFilesystem` which serves Stat()/Lstat() by returning the global index entry from the database, and serves Open() calls by reading the file from a remote peer and buffering it in-memory. Obviously this is inefficient for general usage, but it's perfectly fine for ignore files. The ignore loader then gets a layered view of the local filesystem + a remote filesystem that serves files that don't exist locally, via the new `LayeredFilesystem` abstraction. Signed-off-by: Jakob Borg <jakob@kastelo.net>

Signed-off-by: Jakob Borg <jakob@kastelo.net>

acolomb

Beautiful architecture! I wonder how much overlap there is with the remote streaming access implemented by @pixelspark? Maybe the two approaches can converge in the long term?

Some dumb questions and ideas came to mind during review, ignore at will.

acolomb · 2026-04-27T19:52:06Z

+// can supply paths missing from higher ones; any other error is surfaced
+// immediately without consulting further layers.
+//
+// Walks, globs and directory listings are not merged across layers — the


Not quite what the code does currently. Walk targets only the top layer.

acolomb · 2026-04-27T19:57:33Z

+func (f *layeredFilesystem) Watch(path string, ignore Matcher, ctx context.Context, ignorePerms bool) (<-chan Event, <-chan error, error) {
+	return f.top().Watch(path, ignore, ctx, ignorePerms)
+}
+
+func (f *layeredFilesystem) SameFile(a, b FileInfo) bool {
+	return f.top().SameFile(a, b)
+}


I see how these don't fit the common signature assumed in readCascade(), but at least for SameFile, wouldn't it make sense to not only consider the top layer? Feels like this touches an invariant that may cause subtle bugs later on.

We'd have to change the signature to enable that, as the current return doesn't expose the reason for a false return. Additionally, this is only ever used as a check prior to doing a write operation, which can only affect the top layer...

acolomb · 2026-04-27T20:04:36Z

+
+func (f *remoteFilesystem) URI() string { return "remote://" + f.folderId }
+
+func (f *remoteFilesystem) Options() []Option { return nil }


Could return the openTimeout as an option here?

Kinda, but not currently as it's not taken as an Option as that isn't how the remotefs is instantiated. Some liberties taken here since this is not an fs that would/could ever back an actual folder in its entirety.

acolomb · 2026-04-27T20:24:17Z

+}
+
+func (*remoteFilesystem) GetXattr(_ string, _ XattrFilter) ([]protocol.Xattr, error) {
+	return nil, errRemoteNotImpl


We do store the PlatformData in the DB, is there a good reason to not even try returning it here? Except being additional work for no immediate benefit.

Yeah, that, effectively. We don't need it, so it would just be unused.

Then I guess a comment to that effect would help anybody who might later need it and wonders why it doesn't work.

acolomb · 2026-04-27T20:33:03Z

+	return nil, nil, ErrWatchNotSupported
+}
+
+func (*remoteFilesystem) SameFile(_, _ FileInfo) bool { return false }


This could be implemented with little effort by exposing a primary key (sequence) comparison on the folder DB. Or even simpler, just a string comparison on the name, since the file_names has a name TEXT NOT NULL UNIQUE COLLATE BINARY guarantee.

As above, this is only ever used for handling utf-8 normalisation renames.

acolomb · 2026-04-27T20:38:22Z

+
+func (fi *remoteFileInfo) IsRegular() bool { return fi.info.Type == protocol.FileInfoTypeFile }
+
+func (fi *remoteFileInfo) IsSymlink() bool { return fi.info.IsSymlink() }


Doesn't this contradict SymlinksSupported above always returning false? Or what exactly is the semantic expectation implied by SymlinksSupported?

Possibly... in fact, looking at it, I don't see anywhere we actually call SymlinksSupported() ever, so that may be a historical artifact to remove. Will look into that separately...

Yes SymlinksSupported() is never called.

pixelspark · 2026-04-27T21:26:14Z

@acolomb thanks for the tag!

First of all, this PR looks good and should scratch an itch for a few power users. I envision the next step could be to allow provisioning a 'global' .stignore (i.e. have a configuration setting that tells Syncthing that instead of looking at a local .stignore, it should assume the contents had been #include .stglobalignore, and that file could then exist in the global tree).

On the remote FS part: Synctrain indeed uses the same RequestGlobal / GetGlobalAvailability primitives used here (albeit through the Internals interface) to fetch (parts of) files on demand using BEP. If the 'remote FS' interface ever becomes (informal, unstable) API, it could be used by Synctrain. In the current form however, the existing primitives are much more useful, as they allow Synctrain to implement various improvements specific to its use cases.

A primary use case in Synctrain is on-demand video streaming from an (incompletely synced, because .stignore contains *) Syncthing folder on an iPhone. The downstream interface for the stream is an HTTP server serving range requests. The native iOS video player will receive a localhost URL and make periodic range requests to it. These byte ranges are translated to ranges of blocks, which are then either loaded from the local copy of the file, an in-memory LRU block cache, or requested using RequestGlobal.

For the remote fetch, we have to deal with the fact that mobile connectivity can be highly dynamic. If you happen to be on a Wi-Fi LAN with a peer that has your video, you want the app to download from that peer instead of through a 3G connection to a peer far away. If a peer suddenly becomes slow or unreachable while streaming, the app should switch over to another one. Also latency tends to be higher on mobile, so you want to make an educated guess about which peer can serve you best, or you'll have to wait needlessly long before the video starts.

The current implementation (which I call the 'mini puller') basically first sorts peers by their measured latency (measured periodically by the app by making requests to invalid blocks to each peer, and measuring the time it takes for the error to come back :-)). For each download, it keeps track of its 'experience' with downloading blocks from each peer (which can be either success, fail or unknown). It will try to keep downloading from peers that were succesful before, if that fails it will try an unknown one, and finally it will retry the bad peers (before the cycle restarts). Then there is some timeout/retry logic. (I am considering adding e.g. 'race' logic, where the app attempts to download a block from multiple peers, and then goes with whatever peer is the first to deliver a block. Still have to experiment a bit).

(Edit: as you can see there is also some logic for pipelining block download. This is used when the user wants to download a file on-demand in full.)

If such logic is useful to have in Synctrain core, feel free to take some inspiration.

Signed-off-by: Jakob Borg <jakob@kastelo.net>

* upstream/main: chore(gui, man, authors): update docs, translations, and contributors

github-actions Bot added the enhancement New features or improvements of some kind, as opposed to a problem (bug) label Apr 25, 2026

calmh changed the title ~~feat: load remote ignore patterns~~ feat: load remote ignore patterns (fixes #9352) Apr 25, 2026

calmh mentioned this pull request Apr 25, 2026

Auto-sync included ignore file #9352

Open

calmh force-pushed the jb/remotefs branch from 47b7cfe to 6a5773d Compare April 26, 2026 13:12

calmh force-pushed the jb/remotefs branch from 6a5773d to de22d09 Compare April 26, 2026 13:13

calmh added 2 commits April 26, 2026 15:17

wip

cf88cea

Signed-off-by: Jakob Borg <jakob@kastelo.net>

wip

6fd3152

Signed-off-by: Jakob Borg <jakob@kastelo.net>

acolomb reviewed Apr 27, 2026

View reviewed changes

calmh added 2 commits April 28, 2026 08:52

wip

a14e038

Signed-off-by: Jakob Borg <jakob@kastelo.net>

Merge remote-tracking branch 'upstream/main' into jb/remotefs

59d9567

* upstream/main: chore(gui, man, authors): update docs, translations, and contributors

pixelspark mentioned this pull request May 5, 2026

chore(model): improve block pull scheduler #10678

Draft

cravl-dev mentioned this pull request Aug 19, 2026

Ignore marker in directories #8982

Open


		func (f *remoteFilesystem) URI() string { return "remote://" + f.folderId }

		func (f *remoteFilesystem) Options() []Option { return nil }


		func (fi *remoteFileInfo) IsRegular() bool { return fi.info.Type == protocol.FileInfoTypeFile }

		func (fi *remoteFileInfo) IsSymlink() bool { return fi.info.IsSymlink() }

Uh oh!

Conversation

calmh commented Apr 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tomasz1986 commented Apr 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

calmh commented Apr 25, 2026

Uh oh!

acolomb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pixelspark commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

calmh commented Apr 25, 2026 •

edited

Loading

tomasz1986 commented Apr 25, 2026 •

edited

Loading

pixelspark commented Apr 27, 2026 •

edited

Loading