FSharpChecker.Create takes three options that decide how much of a background check the background compiler keeps. The transparent compiler stores two of them and never reads them, and reads the third for one field only:
keepAllBackgroundSymbolUses and enableBackgroundItemKeyStoreAndSemanticClassification are constructor parameters (L418-L419) and have no other use in TransparentCompiler.fs.keepAllBackgroundResolutions only picks tcEnvAtEndOfFile (L1489).
ComputeTcIntermediate always creates a TcResultsSinkImpl (L1423) and keeps it in the TcIntermediate it caches (L79-L92). So every file that was type checked, also the files that were only checked because a later file depends on them, keeps all its name resolutions for as long as its TcIntermediate is cached. ComputeParseAndCheckAllFilesInProject also collects the sinks of all files of the project (collectSinks = true, L1758), and that is the path that produces the assembly data of an in-memory project reference.
With the background compiler, keepAllBackgroundResolutions = false and keepAllBackgroundSymbolUses = false drop the resolutions and symbol uses of the files it checked in the background.
Measured
FsAutoComplete with FCS 43.12.201, transparent compiler, default cache sizes, on the Fantomas solution. Session: open two files of Fantomas.Core, hover and navigate, then one Find References on SyntaxOak.Expr (672 references in 9 files). Dominator tree of a heap dump taken at the end:
|
Retained |
| Live heap |
689 MB |
CompilerCaches |
386 MB |
TcIntermediate cache, 108 entries |
73 MB |
The 108 TcResultsSinkImpl in those entries |
34 MB |
The same session with the options turned off in FsAutoComplete, one run each:
| Options |
RSS after Find References |
All three true (FsAutoComplete's default) |
1,112 MB |
keepAllBackgroundResolutions = false |
1,117 MB |
keepAllBackgroundSymbolUses = false, enableBackgroundItemKeyStoreAndSemanticClassification = false |
1,134 MB |
Run-to-run noise is about ±25 MB, so none of them changes anything. The links above point to main; 43.12.201 has the same code in these places.
Proposal
- Honour
keepAllBackgroundSymbolUses = false in ComputeParseAndCheckAllFilesInProject: do not collect the sinks. The background compiler does not keep the symbol uses either.
- Honour
keepAllBackgroundResolutions = false: keep the sink only for the file that was asked for. ComputeTcLastFile only uses the sink of the last file, so the TcIntermediate of the files before it does not need one. The cost is that opening one of those files later type checks it again, as with the background compiler.
enableBackgroundItemKeyStoreAndSemanticClassification: the transparent compiler builds both on demand from the sink, so the option means nothing there. Say so in its XML doc.
Expected gain: up to the 34 MB above for this session, more where in-memory project references are type checked for their assembly data. I have not measured that case.
Related: #8021 (the same data structure in the background compiler).
FSharpChecker.Createtakes three options that decide how much of a background check the background compiler keeps. The transparent compiler stores two of them and never reads them, and reads the third for one field only:keepAllBackgroundSymbolUsesandenableBackgroundItemKeyStoreAndSemanticClassificationare constructor parameters (L418-L419) and have no other use inTransparentCompiler.fs.keepAllBackgroundResolutionsonly pickstcEnvAtEndOfFile(L1489).ComputeTcIntermediatealways creates aTcResultsSinkImpl(L1423) and keeps it in theTcIntermediateit caches (L79-L92). So every file that was type checked, also the files that were only checked because a later file depends on them, keeps all its name resolutions for as long as itsTcIntermediateis cached.ComputeParseAndCheckAllFilesInProjectalso collects the sinks of all files of the project (collectSinks = true, L1758), and that is the path that produces the assembly data of an in-memory project reference.With the background compiler,
keepAllBackgroundResolutions = falseandkeepAllBackgroundSymbolUses = falsedrop the resolutions and symbol uses of the files it checked in the background.Measured
FsAutoComplete with FCS 43.12.201, transparent compiler, default cache sizes, on the Fantomas solution. Session: open two files of Fantomas.Core, hover and navigate, then one Find References on
SyntaxOak.Expr(672 references in 9 files). Dominator tree of a heap dump taken at the end:CompilerCachesTcIntermediatecache, 108 entriesTcResultsSinkImplin those entriesThe same session with the options turned off in FsAutoComplete, one run each:
true(FsAutoComplete's default)keepAllBackgroundResolutions = falsekeepAllBackgroundSymbolUses = false,enableBackgroundItemKeyStoreAndSemanticClassification = falseRun-to-run noise is about ±25 MB, so none of them changes anything. The links above point to main; 43.12.201 has the same code in these places.
Proposal
keepAllBackgroundSymbolUses = falseinComputeParseAndCheckAllFilesInProject: do not collect the sinks. The background compiler does not keep the symbol uses either.keepAllBackgroundResolutions = false: keep the sink only for the file that was asked for.ComputeTcLastFileonly uses the sink of the last file, so theTcIntermediateof the files before it does not need one. The cost is that opening one of those files later type checks it again, as with the background compiler.enableBackgroundItemKeyStoreAndSemanticClassification: the transparent compiler builds both on demand from the sink, so the option means nothing there. Say so in its XML doc.Expected gain: up to the 34 MB above for this session, more where in-memory project references are type checked for their assembly data. I have not measured that case.
Related: #8021 (the same data structure in the background compiler).