Skip to content

Commit b7e5325

Browse files
NatElkinsNatElkins
committed
Add Roslyn-format EnC CustomDebugInformation codec and portable PDB method CDI emission
Adds an internal AbstractIL module implementing, byte for byte, the three Portable PDB CustomDebugInformation blob formats Roslyn persists per method for Edit and Continue (EnC Local Slot Map, EnC Lambda and Closure Map, EnC State Machine State Map), with serializers, deserializers, a portable PDB read-back helper, and an occurrence-key packing helper for deterministic syntax-offset slots. Plumbs an optional methodCustomDebugInfoRows side channel through the IL binary writer options into the portable PDB generator so a compilation can attach CDI rows to named methods. Names that do not identify exactly one method row are dropped. All existing writer call sites pass an empty map, so emitted PDBs are byte-identical to before. No in-tree caller populates the map yet; the consumer is the F# hot reload work in #19941, following the same pattern as #20017 (land isolated, test-covered infrastructure first, wire the feature later). Tests: blob round-trips, Roslyn golden-byte encodings, cross-validation against CDI blobs emitted by a real Roslyn compilation, fail-closed occurrence-key packing (including an int32-overflow regression where a wrapped negative key previously escaped the bound check), and end-to-end synthetic PDB emission proving correct MethodDef parenting, zero rows for an empty map, and no rows for absent or ambiguous names.
1 parent 4d06bd1 commit b7e5325

12 files changed

Lines changed: 1467 additions & 22 deletions

File tree

‎docs/release-notes/.FSharp.Compiler.Service/11.0.100.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@
117117
* Debug: fix if and match condition sequence points ([PR #19932](https://github.com/dotnet/fsharp/pull/19932))
118118
* Checker: recover on checking language version ([PR ##19970](https://github.com/dotnet/fsharp/pull/19970))
119119
* Implied argument names for function-to-delegate coercions now fall back to the delegate's `Invoke` parameter names when the function has no recoverable names (e.g. a partial application like `System.Func<int, int>((+) 1)`), instead of synthetic `delegateArg0`, `delegateArg1`, … names. ([PR #20001](https://github.com/dotnet/fsharp/pull/20001))
120+
* Add Roslyn-format EnC CustomDebugInformation codec and portable PDB method CDI emission support to AbstractIL. ([PR #20018](https://github.com/dotnet/fsharp/pull/20018))
120121

121122
### Improved
122123

‎src/Compiler/AbstractIL/EncMethodDebugInformation.fs‎

Lines changed: 557 additions & 0 deletions
Large diffs are not rendered by default.

‎src/Compiler/AbstractIL/EncMethodDebugInformation.fsi‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
/// Edit-and-Continue method debug information blobs.
2+
///
3+
/// This module replicates, byte for byte, the three Portable-PDB CustomDebugInformation
4+
/// blob formats Roslyn persists per method to support Edit and Continue
5+
/// (roslyn/src/Compilers/Core/Portable/Emit/EditAndContinueMethodDebugInformation.cs):
6+
///
7+
/// - EnC Local Slot Map (kind 755F52A8-91C5-45BE-B4B8-209571E552BD)
8+
/// - EnC Lambda and Closure Map (kind A643004C-0240-496F-A783-30D64F4979DE)
9+
/// - EnC State Machine State Map (kind 8B78CD68-2EDE-420B-980B-E15884B8AAA3)
10+
///
11+
/// (GUIDs: roslyn/src/Dependencies/CodeAnalysis.Debugging/PortableCustomDebugInfoKinds.cs.)
12+
///
13+
/// All multi-byte integers use the ECMA-335 compressed unsigned/signed encodings via
14+
/// System.Reflection.Metadata's BlobBuilder.WriteCompressedInteger /
15+
/// WriteCompressedSignedInteger and BlobReader.ReadCompressedInteger /
16+
/// ReadCompressedSignedInteger, exactly as Roslyn writes/reads them.
17+
///
18+
/// Every "syntax offset" slot in these blobs is an opaque, caller-defined integer key
19+
/// (Roslyn: the syntax offset of the lambda/closure/state-machine-suspension syntax
20+
/// node). This module does not require the key to be a source offset; it only requires
21+
/// determinism across generations. tryEncodeOccurrenceKey/decodeOccurrenceKey provide one
22+
/// reusable way to pack a short (depth <= 2) ordinal chain into such a key.
23+
module internal FSharp.Compiler.AbstractIL.EncMethodDebugInformation
24+
25+
/// Portable-PDB CustomDebugInformation kind GUIDs for the EnC blobs, copied verbatim
26+
/// from roslyn/src/Dependencies/CodeAnalysis.Debugging/PortableCustomDebugInfoKinds.cs.
27+
[<RequireQualifiedAccess>]
28+
module PortableCustomDebugInfoKinds =
29+
30+
/// EnC Local Slot Map CDI kind.
31+
val encLocalSlotMap: System.Guid
32+
33+
/// EnC Lambda and Closure Map CDI kind.
34+
val encLambdaAndClosureMap: System.Guid
35+
36+
/// EnC State Machine State Map CDI kind.
37+
val encStateMachineStateMap: System.Guid
38+
39+
/// Closure ordinal of a lambda that is lowered to a static (non-capturing) method.
40+
/// Mirrors Roslyn's LambdaDebugInfo.StaticClosureOrdinal.
41+
[<Literal>]
42+
val StaticClosureOrdinal: int = -1
43+
44+
/// Closure ordinal of a lambda closed over the 'this' pointer only.
45+
/// Mirrors Roslyn's LambdaDebugInfo.ThisOnlyClosureOrdinal.
46+
[<Literal>]
47+
val ThisOnlyClosureOrdinal: int = -2
48+
49+
/// Smallest valid closure ordinal. Mirrors Roslyn's LambdaDebugInfo.MinClosureOrdinal.
50+
[<Literal>]
51+
val MinClosureOrdinal: int = -2
52+
53+
/// Method ordinal of a method that has no lambda map (an empty blob decodes to this).
54+
/// Mirrors Roslyn's DebugId.UndefinedOrdinal.
55+
[<Literal>]
56+
val UndefinedMethodOrdinal: int = -1
57+
58+
/// Largest synthesized-local kind serializable in the slot map: the kind is stored as
59+
/// (kind + 1) in bits 0-5 of the leading byte (bit 6 is unused, bit 7 flags a trailing ordinal), and
60+
/// Roslyn's reader recovers it with mask 0x3F, so only kinds 0..0x3E round-trip.
61+
[<Literal>]
62+
val MaxSerializableLocalKind: int = 0x3E
63+
64+
/// One slot in the EnC Local Slot Map: the local variable layout of a method body,
65+
/// recorded so a later generation can map its locals onto the same slot indices.
66+
[<RequireQualifiedAccess>]
67+
type EncLocalSlotInfo =
68+
/// A short-lived lowering temp: serialized as the single byte 0x00, carrying no
69+
/// identity (a later generation never reuses it).
70+
| Temp
71+
72+
/// A long-lived synthesized local.
73+
/// kind: synthesized-local kind (Roslyn SynthesizedLocalKind value, 0..MaxSerializableLocalKind;
74+
/// 0 = user-defined local).
75+
/// syntaxOffset: caller-defined key of the declaring occurrence
76+
/// (Roslyn: syntax offset of the local's declarator).
77+
/// ordinal: zero-based disambiguator among slots sharing the same kind and offset (>= 0).
78+
| Slot of kind: int * syntaxOffset: int * ordinal: int
79+
80+
/// One closure scope in the EnC Lambda and Closure Map. The closure's ordinal is its
81+
/// index in EncMethodDebugInformation.Closures; lambdas reference closures by that index.
82+
type EncClosureInfo =
83+
{
84+
/// Caller-defined key (Roslyn: syntax offset of the scope owning the closure).
85+
SyntaxOffset: int
86+
}
87+
88+
/// One lambda in the EnC Lambda and Closure Map.
89+
type EncLambdaInfo =
90+
{
91+
/// Caller-defined key (Roslyn: syntax offset of the lambda body).
92+
SyntaxOffset: int
93+
/// Index into EncMethodDebugInformation.Closures of the closure holding the
94+
/// lambda's captures, or StaticClosureOrdinal / ThisOnlyClosureOrdinal.
95+
ClosureOrdinal: int
96+
}
97+
98+
/// One suspension point in the EnC State Machine State Map.
99+
type EncStateMachineStateInfo =
100+
{
101+
/// State machine state number assigned to the suspension point (may be negative:
102+
/// Roslyn uses negative numbers for increasing-iteration finalize states).
103+
StateNumber: int
104+
/// Caller-defined key (Roslyn: syntax offset of the await/yield syntax node).
105+
SyntaxOffset: int
106+
}
107+
108+
/// Debugging information associated with a method, persisted by the compiler in the
109+
/// Portable PDB to support Edit and Continue. Mirrors Roslyn's
110+
/// EditAndContinueMethodDebugInformation.
111+
type EncMethodDebugInformation =
112+
{
113+
/// Ordinal of the method within its generation (>= -1; UndefinedMethodOrdinal when absent).
114+
MethodOrdinal: int
115+
/// Local slot layout, in slot-index order (EnC Local Slot Map).
116+
LocalSlots: EncLocalSlotInfo list
117+
/// Closure scopes, in ordinal order (EnC Lambda and Closure Map).
118+
Closures: EncClosureInfo list
119+
/// Lambdas, in ordinal order (EnC Lambda and Closure Map).
120+
Lambdas: EncLambdaInfo list
121+
/// State machine suspension points (EnC State Machine State Map).
122+
StateMachineStates: EncStateMachineStateInfo list
123+
}
124+
125+
/// An empty map (no slots, lambdas, closures or states; undefined method ordinal).
126+
static member Empty: EncMethodDebugInformation
127+
128+
/// Packs an ordinal chain (root-first enclosing ordinals, ending with the innermost
129+
/// ordinal) into a deterministic int suitable for a "syntax offset" blob slot. Fails
130+
/// closed (None) past the limits: chains deeper than 2, ordinals > 0xFFFF, or keys
131+
/// exceeding the compressed-integer budget.
132+
val tryEncodeOccurrenceKey: ordinalChain: int list -> int option
133+
134+
/// Unpacks an occurrence key produced by tryEncodeOccurrenceKey back into its
135+
/// root-first ordinal chain.
136+
val decodeOccurrenceKey: key: int -> int list
137+
138+
/// Serializes the EnC Local Slot Map blob for 'info', byte-for-byte as Roslyn's
139+
/// SerializeLocalSlots. Returns the empty array when there are no slots (no CDI row
140+
/// should be emitted then).
141+
val serializeLocalSlots: info: EncMethodDebugInformation -> byte[]
142+
143+
/// Deserializes an EnC Local Slot Map blob, byte-for-byte as Roslyn's UncompressSlotMap.
144+
/// An empty (or null) blob yields no slots.
145+
val deserializeLocalSlots: blob: byte[] -> EncLocalSlotInfo list
146+
147+
/// Serializes the EnC Lambda and Closure Map blob for 'info', byte-for-byte as Roslyn's
148+
/// SerializeLambdaMap. Returns the empty array when there are no lambdas and no closures
149+
/// (Roslyn's MetadataWriter skips the CDI row in that case; note the method ordinal is
150+
/// then not persisted and decodes back as UndefinedMethodOrdinal).
151+
val serializeLambdaMap: info: EncMethodDebugInformation -> byte[]
152+
153+
/// Deserializes an EnC Lambda and Closure Map blob, byte-for-byte as Roslyn's
154+
/// UncompressLambdaMap. An empty (or null) blob yields (UndefinedMethodOrdinal, [], []).
155+
val deserializeLambdaMap: blob: byte[] -> int * EncClosureInfo list * EncLambdaInfo list
156+
157+
/// Serializes the EnC State Machine State Map blob for 'info', byte-for-byte as
158+
/// Roslyn's SerializeStateMachineStates: entries are sorted by syntax offset (stably,
159+
/// preserving relative order of equal offsets, which encodes the per-offset relative
160+
/// ordinal). Returns the empty array when there are no states (no CDI row then).
161+
val serializeStateMachineStates: info: EncMethodDebugInformation -> byte[]
162+
163+
/// Deserializes an EnC State Machine State Map blob, byte-for-byte as Roslyn's
164+
/// UncompressStateMachineStates (including the ordered-by-offset and <= 256-per-offset
165+
/// validations). An empty (or null) blob yields no states.
166+
val deserializeStateMachineStates: blob: byte[] -> EncStateMachineStateInfo list
167+
168+
/// Deserializes EnC method debug information from the three blobs (any of which may be
169+
/// null or empty). Mirrors Roslyn's EditAndContinueMethodDebugInformation.Create.
170+
val deserialize:
171+
slotMapBlob: byte[] -> lambdaMapBlob: byte[] -> stateMachineStateMapBlob: byte[] -> EncMethodDebugInformation
172+
173+
/// Decodes every method-level EnC CustomDebugInformation row of a portable PDB image into
174+
/// per-method EnC debug information, keyed by MethodDef token (0x06xxxxxx). The CDI parent
175+
/// of the EnC rows is always a MethodDef handle, so token keying is unambiguous.
176+
/// Fail safe: a null/empty or non-PDB image yields the empty map, and a method whose
177+
/// blobs do not decode is omitted rather than guessed.
178+
val readEncMethodDebugInfoFromPortablePdb: pdbBytes: byte[] -> Map<int, EncMethodDebugInformation>

‎src/Compiler/AbstractIL/ilwrite.fs‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3859,7 +3859,10 @@ type options =
38593859
referenceAssemblyOnly: bool
38603860
referenceAssemblyAttribOpt: ILAttribute option
38613861
referenceAssemblySignatureHash : int option
3862-
pathMap: PathMap }
3862+
pathMap: PathMap
3863+
/// Per-method EnC CustomDebugInformation rows for the portable PDB writer, keyed by
3864+
/// IL method name. Empty for ordinary compiles, so flag-off output stays byte-identical.
3865+
methodCustomDebugInfoRows: Map<string, PdbMethodCustomDebugInfo list> }
38633866

38643867
let writeBinaryAux (stream: Stream, options: options, modul, normalizeAssemblyRefs) =
38653868

@@ -4022,7 +4025,7 @@ let writeBinaryAux (stream: Stream, options: options, modul, normalizeAssemblyRe
40224025
match options.pdbfile, options.portablePDB with
40234026
| Some _, true ->
40244027
let pdbInfo =
4025-
generatePortablePdb options.embedAllSource options.embedSourceList options.sourceLink options.checksumAlgorithm pdbData options.pathMap
4028+
generatePortablePdb options.embedAllSource options.embedSourceList options.sourceLink options.checksumAlgorithm pdbData options.pathMap options.methodCustomDebugInfoRows
40264029

40274030
if options.embeddedPDB then
40284031
let uncompressedLength, contentId, stream, algorithmName, checkSum = pdbInfo

‎src/Compiler/AbstractIL/ilwrite.fsi‎

Lines changed: 23 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -9,24 +9,29 @@ open FSharp.Compiler.AbstractIL.ILPdbWriter
99
open FSharp.Compiler.AbstractIL.StrongNameSign
1010

1111
type options =
12-
{ ilg: ILGlobals
13-
outfile: string
14-
pdbfile: string option
15-
portablePDB: bool
16-
embeddedPDB: bool
17-
embedAllSource: bool
18-
embedSourceList: string list
19-
allGivenSources: ILSourceDocument list
20-
sourceLink: string
21-
checksumAlgorithm: HashAlgorithm
22-
signer: ILStrongNameSigner option
23-
emitTailcalls: bool
24-
deterministic: bool
25-
dumpDebugInfo: bool
26-
referenceAssemblyOnly: bool
27-
referenceAssemblyAttribOpt: ILAttribute option
28-
referenceAssemblySignatureHash: int option
29-
pathMap: PathMap }
12+
{
13+
ilg: ILGlobals
14+
outfile: string
15+
pdbfile: string option
16+
portablePDB: bool
17+
embeddedPDB: bool
18+
embedAllSource: bool
19+
embedSourceList: string list
20+
allGivenSources: ILSourceDocument list
21+
sourceLink: string
22+
checksumAlgorithm: HashAlgorithm
23+
signer: ILStrongNameSigner option
24+
emitTailcalls: bool
25+
deterministic: bool
26+
dumpDebugInfo: bool
27+
referenceAssemblyOnly: bool
28+
referenceAssemblyAttribOpt: ILAttribute option
29+
referenceAssemblySignatureHash: int option
30+
pathMap: PathMap
31+
/// Per-method EnC CustomDebugInformation rows for the portable PDB writer, keyed by
32+
/// IL method name. Empty for ordinary compiles, so flag-off output stays byte-identical.
33+
methodCustomDebugInfoRows: Map<string, PdbMethodCustomDebugInfo list>
34+
}
3035

3136
/// Write a binary to the file system.
3237
val WriteILBinaryFile: options: options * inputModule: ILModuleDef * (ILAssemblyRef -> ILAssemblyRef) -> unit

‎src/Compiler/AbstractIL/ilwritepdb.fs‎

Lines changed: 53 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,10 @@ type PdbMethodData =
118118
DebugPoints: PdbDebugPoint array
119119
}
120120

121+
/// A pre-serialized CustomDebugInformation row (kind GUID + blob) to attach to a method
122+
/// definition row in the portable PDB.
123+
type PdbMethodCustomDebugInfo = { KindGuid: Guid; Blob: byte[] }
124+
121125
module SequencePoint =
122126
let orderBySource sp1 sp2 =
123127
let c1 = compare sp1.Document sp2.Document
@@ -337,7 +341,15 @@ let scopeSorter (scope1: PdbMethodScope) (scope2: PdbMethodScope) =
337341
0
338342

339343
type PortablePdbGenerator
340-
(embedAllSource: bool, embedSourceList: string list, sourceLink: string, checksumAlgorithm, info: PdbData, pathMap: PathMap) =
344+
(
345+
embedAllSource: bool,
346+
embedSourceList: string list,
347+
sourceLink: string,
348+
checksumAlgorithm,
349+
info: PdbData,
350+
pathMap: PathMap,
351+
methodCustomDebugInfoRows: Map<string, PdbMethodCustomDebugInfo list>
352+
) =
341353

342354
// Deterministic: build the Document table in a stable order by mapped file path,
343355
// but preserve the original-document-index -> handle mapping by filename.
@@ -488,6 +500,27 @@ type PortablePdbGenerator
488500
let moduleImportScopeHandle = MetadataTokens.ImportScopeHandle(1)
489501
let importScopesTable = Dictionary<PdbImports, ImportScopeHandle>()
490502

503+
// Per-method CustomDebugInformation rows keyed by IL method name. Names that match
504+
// more than one method row (overloads, same name on different types) fail closed and
505+
// attach nothing, so a row can never land on the wrong method.
506+
let methodCustomDebugInfoByName =
507+
if Map.isEmpty methodCustomDebugInfoRows then
508+
methodCustomDebugInfoRows
509+
else
510+
let nameCounts = Dictionary<string, int>()
511+
512+
for minfo in info.Methods do
513+
nameCounts[minfo.MethName] <-
514+
match nameCounts.TryGetValue minfo.MethName with
515+
| true, count -> count + 1
516+
| _ -> 1
517+
518+
methodCustomDebugInfoRows
519+
|> Map.filter (fun methName _ ->
520+
match nameCounts.TryGetValue methName with
521+
| true, 1 -> true
522+
| _ -> false)
523+
491524
let serializeImport (writer: BlobBuilder) (import: PdbImport) =
492525
match import with
493526
// We don't yet emit these kinds of imports
@@ -777,6 +810,23 @@ type PortablePdbGenerator
777810

778811
metadata.AddMethodDebugInformation(docHandle, sequencePointBlob) |> ignore
779812

813+
// MetadataBuilder sorts the CustomDebugInformation table by parent at serialize
814+
// time, so adding rows in method order here is safe.
815+
match Map.tryFind minfo.MethName methodCustomDebugInfoByName with
816+
| Some cdiRows ->
817+
// MethToken is the uncoded token (0x06 <<< 24 ||| rid); the handle needs the rid.
818+
let methodHandle =
819+
MetadataTokens.MethodDefinitionHandle(minfo.MethToken &&& 0x00FFFFFF)
820+
821+
for cdiRow in cdiRows do
822+
metadata.AddCustomDebugInformation(
823+
MethodDefinitionHandle.op_Implicit methodHandle,
824+
metadata.GetOrAddGuid cdiRow.KindGuid,
825+
metadata.GetOrAddBlob cdiRow.Blob
826+
)
827+
|> ignore
828+
| None -> ()
829+
780830
match minfo.RootScope with
781831
| None -> ()
782832
| Some scope -> writeMethodScopes minfo.MethToken scope
@@ -831,9 +881,10 @@ let generatePortablePdb
831881
checksumAlgorithm
832882
(info: PdbData)
833883
(pathMap: PathMap)
884+
(methodCustomDebugInfoRows: Map<string, PdbMethodCustomDebugInfo list>)
834885
=
835886
let generator =
836-
PortablePdbGenerator(embedAllSource, embedSourceList, sourceLink, checksumAlgorithm, info, pathMap)
887+
PortablePdbGenerator(embedAllSource, embedSourceList, sourceLink, checksumAlgorithm, info, pathMap, methodCustomDebugInfoRows)
837888

838889
generator.Emit()
839890

‎src/Compiler/AbstractIL/ilwritepdb.fsi‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,12 @@ type PdbMethodData =
6767
DebugRange: (PdbSourceLoc * PdbSourceLoc) option
6868
DebugPoints: PdbDebugPoint[] }
6969

70+
/// A pre-serialized CustomDebugInformation row to attach to a method definition row in
71+
/// the portable PDB (kind GUID + blob). Supplied by the compiler as a side channel keyed
72+
/// by IL method name. The writer attaches the rows only when the name identifies exactly
73+
/// one method row (fail closed on ambiguity).
74+
type PdbMethodCustomDebugInfo = { KindGuid: System.Guid; Blob: byte[] }
75+
7076
[<NoEquality; NoComparison>]
7177
type PdbData =
7278
{
@@ -109,6 +115,7 @@ val generatePortablePdb:
109115
checksumAlgorithm: HashAlgorithm ->
110116
info: PdbData ->
111117
pathMap: PathMap ->
118+
methodCustomDebugInfoRows: Map<string, PdbMethodCustomDebugInfo list> ->
112119
int64 * BlobContentId * MemoryStream * string * byte[]
113120

114121
val compressPortablePdbStream: stream: MemoryStream -> MemoryStream

‎src/Compiler/Driver/fsc.fs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1149,6 +1149,7 @@ let main6
11491149
referenceAssemblyAttribOpt = referenceAssemblyAttribOpt
11501150
referenceAssemblySignatureHash = refAssemblySignatureHash
11511151
pathMap = tcConfig.pathMap
1152+
methodCustomDebugInfoRows = Map.empty
11521153
},
11531154
ilxMainModule,
11541155
normalizeAssemblyRefs
@@ -1180,6 +1181,7 @@ let main6
11801181
referenceAssemblyAttribOpt = None
11811182
referenceAssemblySignatureHash = None
11821183
pathMap = tcConfig.pathMap
1184+
methodCustomDebugInfoRows = Map.empty
11831185
},
11841186
ilxMainModule,
11851187
normalizeAssemblyRefs

‎src/Compiler/FSharp.Compiler.Service.fsproj‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,8 @@
236236
<Compile Include="AbstractIL\ilbinary.fs" />
237237
<Compile Include="AbstractIL\ilread.fsi" />
238238
<Compile Include="AbstractIL\ilread.fs" />
239+
<Compile Include="AbstractIL\EncMethodDebugInformation.fsi" />
240+
<Compile Include="AbstractIL\EncMethodDebugInformation.fs" />
239241
<Compile Include="AbstractIL\ilwritepdb.fsi" />
240242
<Compile Include="AbstractIL\ilwritepdb.fs" />
241243
<Compile Include="AbstractIL\ilwrite.fsi" />

0 commit comments

Comments
 (0)