Caches.Cache: its finalizer keeps a dropped check alive for one more full GC, and MailboxProcessor-mode caches are never collected #20756

Cache<'Key, 'Value> in Caches.fs has a finalizer that calls Dispose (L397-L398). I measured two effects of it with FCS 43.12.201. Main has the same code in the places linked below.

1. A dropped check survives one extra full GC

MemoizationTable creates its cache with NoEviction (illib.fs L1103), and InfoReader has 11 of these tables. With NoEviction and Immediate, Dispose has no eviction processor to stop (L308-L309). It only counts the disposal for the metrics (L387-L390). MemoizationTable is not disposable, so the finalizer runs for each table.

An object with a finalizer survives the collection that finds it dead, and so does everything it references, until the finalizer has run and a later collection of its generation happens. A table references the infos it computed, and through them the type checker state. So when a check is dropped, that state survives the first full GC and is promoted to gen 2. A process that goes idle can wait a long time for the next gen 2 GC.

Measured:

• A Cache created through reflection, holding a 50 MB value. Long weak references, which stay set while an object waits for its finalizer:

  Mode	Finalizer	Value alive after one full GC	After WaitForPendingFinalizers and a second full GC
  NoEviction	kept	yes	no
  NoEviction	GC.SuppressFinalize	no	no
  Immediate	kept	yes	no
  Immediate	GC.SuppressFinalize	no	no

• FsAutoComplete on the Fantomas solution, transparent compiler. I called ClearCache for every project, then ran one GC.Collect(2, GCCollectionMode.Aggressive, true, true) and took a heap dump. No root reached 62 MB of the heap, and 44.2 MB of that was reachable only from dead Cache instances. An earlier run left 135 MB unreachable. With GC.WaitForPendingFinalizers() and a second collection, 0.8 MB was left. FsAutoComplete now does that as a workaround.

2. A cache in MailboxProcessor mode is never collected

MailboxProcessor is the default eviction mode (L157). The eviction loop waits in mb.Receive() (L289-L301). FSharp.Core waits there with Async.AwaitWaitHandle, which registers the wait with the thread pool, and the loop references the cache. The root path from a dump:

root Stack System.Threading.PortableThreadPool+WaitThread
  ._registeredWaits -> System.Threading.RegisteredWaitHandle[]
  .[] -> System.Threading.RegisteredWaitHandle
  ._callbackHelper -> System.Threading._ThreadPoolWaitOrTimerCallback
  ._waitOrTimerCallback -> System.Threading.WaitOrTimerCallback
  ._target -> <StartupCode$FSharp-Core>.$Async+AwaitWaitHandle@1918-3
  .ctxt -> Microsoft.FSharp.Control.AsyncActivationContents<System.Boolean>
  .cont@ -> Microsoft.FSharp.Control.AsyncPrimitives+Bind@589<EvictionQueueMessage<...>, ...>
  .ctxt -> Microsoft.FSharp.Control.AsyncActivationContents<EvictionQueueMessage<...>>
  .cont@ -> Microsoft.FSharp.Control.AsyncPrimitives+Bind@589<Unit, EvictionQueueMessage<...>>
  .part2 -> <StartupCode$FSharp-Compiler-Service>.$Caches+processNext@295-1<...>
  .this -> FSharp.Compiler.Caches.Cache<...>

So the cache stays reachable until something disposes it. The finalizer is meant to stop the loop when Dispose was not called, but it can never run. In the reflection test above, a MailboxProcessor cache and its 50 MB value were still alive after three full GCs, with or without GC.SuppressFinalize.

Outside CompilationMode.OneOff, getTypeSubsumptionCache (TypeRelations.fs L42-L49) and getOverloadResolutionCache (OverloadResolutionCache.fs L48-L63) create a cache of this kind for each TcGlobals, kept in a WeakMap. Nothing disposes them, so every TcGlobals leaves its cache and a thread pool wait behind.

Measured: an fsi script creates five FSharpCheckers one after the other, checks a script with a type test with each, and drops them. Then three full GCs, each followed by WaitForPendingFinalizers, and a heap dump:

The background compiler and the transparent compiler give the same counts. The entries do not reference the typed tree, and in FsAutoComplete the caches held 0.3 to 2 MB each. So each TcGlobals leaks little, but nothing ever frees it. My script did not use the overload resolution cache, which is created the same way.

Proposal

• Remove the finalizer. With NoEviction and Immediate it has nothing to clean up, and with MailboxProcessor it cannot run.
• Give the caches that nobody disposes, typeSubsumptionCache and overloadResolutionCache, a lifetime they can end. Two options: use EvictionMode.Immediate for them, or keep the eviction loop from making the cache reachable while it waits. Disposing them would need an owner, and the WeakMap is not one.

#r "nuget: FSharp.Compiler.Service, 43.12.201"
open System
open System.Reflection
open System.Runtime.CompilerServices
open System.Collections.Generic
open Microsoft.FSharp.Reflection

let asm = typeof<FSharp.Compiler.CodeAnalysis.FSharpChecker>.Assembly
let flags = BindingFlags.Public ||| BindingFlags.NonPublic ||| BindingFlags.Static ||| BindingFlags.Instance
let cacheType = asm.GetType("FSharp.Compiler.Caches.Cache`2", true).MakeGenericType(typeof<string>, typeof<byte[]>)
let getDefault = asm.GetType("FSharp.Compiler.Caches.CacheOptions", true).GetMethod("getDefault", flags).MakeGenericMethod(typeof<string>)

[<MethodImpl(MethodImplOptions.NoInlining)>]
let create (mode: string) (suppressFinalizer: bool) =
  let defaults = getDefault.Invoke(null, [| box (EqualityComparer<string>.Default :> IEqualityComparer<string>) |])
  let optionsType = defaults.GetType()
  let fields = FSharpValue.GetRecordFields(defaults, flags)
  let modeCase = FSharpType.GetUnionCases(asm.GetType("FSharp.Compiler.Caches.EvictionMode", true), flags) |> Array.find (fun c -> c.Name = mode)
  fields[FSharpType.GetRecordFields(optionsType, flags) |> Array.findIndex (fun p -> p.Name = "EvictionMode")] <- FSharpValue.MakeUnion(modeCase, [||], flags)
  let cache = (cacheType.GetConstructors(flags) |> Array.exactlyOne).Invoke([| FSharpValue.MakeRecord(optionsType, fields, flags); box (Some "repro") |])
  let value = Array.zeroCreate<byte> (50 * 1024 * 1024)
  cacheType.GetMethod("TryAdd", flags).Invoke(cache, [| box "key"; box value |]) |> ignore
  Threading.Thread.Sleep 200
  if suppressFinalizer then GC.SuppressFinalize cache
  // Long weak reference: it stays set while the object waits for its finalizer
  WeakReference(value, true)

for mode in [ "NoEviction"; "Immediate"; "MailboxProcessor" ] do
  for suppress in [ false; true ] do
    let value = create mode suppress
    GC.Collect(2, GCCollectionMode.Forced, true, true)
    let afterOne = value.IsAlive
    GC.WaitForPendingFinalizers()
    GC.Collect(2, GCCollectionMode.Forced, true, true)
    GC.WaitForPendingFinalizers()
    GC.Collect(2, GCCollectionMode.Forced, true, true)
    printfn "%-16s suppressed %-5b value alive after one GC %-5b after finalizers and two more GCs %b" mode suppress afterOne value.IsAlive

#r "nuget: FSharp.Compiler.Service, 43.12.201"
open System
open System.Runtime.CompilerServices
open FSharp.Compiler.CodeAnalysis
open FSharp.Compiler.Text

let useTransparentCompiler = fsi.CommandLineArgs[1] = "transparent"
let file = IO.Path.Combine(__SOURCE_DIRECTORY__, "Script.fsx")
// The type test makes the checker use the type subsumption cache
let source = SourceText.ofString "let f (o: obj) = match o with :? string as s -> s.Length | :? System.IComparable -> 1 | _ -> 0\nprintfn \"%d\" (f (box \"a\"))"

[<MethodImpl(MethodImplOptions.NoInlining)>]
let check () =
  let checker = FSharpChecker.Create(useTransparentCompiler = useTransparentCompiler)
  let options, _ = checker.GetProjectOptionsFromScript(file, source, assumeDotNetFramework = false) |> Async.RunSynchronously
  let _, answer = checker.ParseAndCheckFileInProject(file, 0, source, options) |> Async.RunSynchronously
  match answer with
  | FSharpCheckFileAnswer.Succeeded _ -> ()
  | FSharpCheckFileAnswer.Aborted -> failwith "aborted"

for _ in 1..5 do check ()
for _ in 1..3 do
  GC.Collect(2, GCCollectionMode.Forced, true, true)
  GC.WaitForPendingFinalizers()
printfn "pid %d" Environment.ProcessId
Threading.Thread.Sleep 120000

Related: #20755 (another thing the transparent compiler keeps in memory, found the same way).