Skip to content

Commit 599f35b

Browse files
committed
[JSC] Wasm stack results should match arguments
https://bugs.webkit.org/show_bug.cgi?id=311838 rdar://174428575 Reviewed by Yusuke Suzuki. Rework how stack results are read after wasm calls so that they match the location of equivalent arguments. This is preliminary work so tail calls don't require whole-module analysis to determine where to restore instance/memory pinned registers. In that world there may or may not be an injected frame between any caller and callee shift the result location by some constant. This makes the current calling convention difficult to maintain as we don't know exactly which of the two offsets to find results. In this patch we go back to the original multi-value calling convention, where arguments and results match. Any caller should copy stack results before restoring SP. Extracting the results differs based on the caller. For BBQ/IPInt we move the stack results to their canonical location. In OMG, createCallPatchpoint stack return value constraints use WarmAny so B3 can place results whereever it wants as they have to be copied anyway. Due to an outstanding issue when graph coloring is enabled where we run out of colors (aka registers) we use still use the argument's stack slot. JSToWasm makes space for registers below the callee frame and fills the register results there then calls to the appropriate operation to build the result Array. Old Wasm calling convention (before this patch): Stack results were offset to the top of the area, so resultK was NOT at the same address as argK: +----------------------------+ | result1 | arg3 | <- resultStackOffset = headerSize + aligned - resultSize +----------------------------+ | result0 | arg2 | <- results shifted, don't align with args +----------------------------+ | | arg1 | +----------------------------+ | | arg0 | <- argStackOffset = headerSize +----------------------------+ | instance | +----------------------------+ | wasmCallee | +----------------------------+ | returnAddress | +----------------------------+ | callerFrame | <- cfr +----------------------------+ New Wasm calling convention (this patch): Arguments and results share the same stack addresses. argK and resultK occupy the same slot. The area is sized to max(args, results): +----------------------------+ higher addresses | ...... | | +----------------------------+ | | arg/result N | | +----------------------------+ | | ... | | +----------------------------+ | | arg/result 1 | | +----------------------------+ | | arg/result 0 | <- headerSize (args & results start here) +----------------------------+ v lower addresses | instance | +----------------------------+ | wasmCallee | +----------------------------+ | returnAddress | +----------------------------+ | callerFrame | <- cfr +----------------------------+ The Nth stack argument and the Nth stack result are at the exact same address. Also, this broke internal builds since a non-ASCII character was added to the IPInt .asm file. Add a check we don't make the same mistake again. No new tests, no behavior change, covered by existing tests. Canonical link: https://commits.webkit.org/310952@main
1 parent d4ddf18 commit 599f35b

8 files changed

Lines changed: 134 additions & 163 deletions

File tree

‎Source/JavaScriptCore/llint/InPlaceInterpreter.asm‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -946,14 +946,10 @@ if ASSERT_ENABLED
946946
clobberVolatileRegisters()
947947
end
948948

949-
# Restore SP
950-
loadp Callee[cfr], ws0 # CalleeBits(JSToWasmCallee*)
951-
unboxWasmCallee(ws0, ws1)
952-
953-
loadi Wasm::JSToWasmCallee::m_frameSize[ws0], ws1
954-
subp cfr, ws1, ws1
955-
move ws1, sp
956-
subp constexpr Wasm::JSToWasmCallee::SpillStackSpaceAligned, sp
949+
# Don't restore SP to original position, stack results live above calleeSP.
950+
# After a tail call the callee's frame may differ, so derive from actual SP.
951+
# Just allocate register spill space below the callee's actual SP.
952+
subp constexpr Wasm::JSToWasmCallee::RegisterStackSpaceAligned, sp
957953

958954
if ASSERT_ENABLED
959955
repeat(ws0, macro (i)

‎Source/JavaScriptCore/llint/InPlaceInterpreter64.asm‎

Lines changed: 7 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -11407,33 +11407,18 @@ _wasm_trampoline_wasm_ipint_call_wide32:
1140711407
_wasm_ipint_call_return_location:
1140811408
_wasm_ipint_call_return_location_wide16:
1140911409
_wasm_ipint_call_return_location_wide32:
11410-
# Restore the stack pointer
11410+
# Compute sc3 (pointing to saved caller info) using the saved SP value,
11411+
# without restoring SP yet. We need callee's SP to read stack results
11412+
# which are at the bottom of the arg/result area (SP + headerSize).
11413+
loadi IPInt::CallReturnMetadata::stackFrameSize[MC], sc3
1141111414
loadp ThisArgumentOffset[cfr], sc0
1141211415
addp cfr, sc0
11413-
move sc0, sp
11414-
11415-
# <first non-arg> <- first_non_arg_addr
11416-
# arg
11417-
# ...
11418-
# arg
11419-
# arg
11420-
# reserved
11421-
# reserved
11422-
# (first_non_arg_addr - cfr), PC
11423-
# (PL - cfr), wasmInstance <- sc3
11424-
# call frame return
11425-
# call frame return
11426-
# call frame
11427-
# call frame
11428-
# call frame
11429-
# call frame <- sp
11430-
11431-
loadi IPInt::CallReturnMetadata::stackFrameSize[MC], sc3
11432-
leap [sp, sc3], sc3
11416+
addp sc0, sc3
1143311417

1143411418
const mintRetSrc = sc1
1143511419
const mintRetDst = sc2
1143611420

11421+
# mintRetSrc: read stack results from the callee's SP (current SP)
1143711422
loadi IPInt::CallReturnMetadata::firstStackResultSPOffset[MC], mintRetSrc
1143811423
advanceMC(IPInt::CallReturnMetadata::resultBytecode)
1143911424
leap [sp, mintRetSrc], mintRetSrc
@@ -11584,12 +11569,10 @@ mintAlign(_end)
1158411569
# return result <- mintRetDst => new SP
1158511570
# (first_non_arg_addr - cfr), PC
1158611571
# (PL - cfr), wasmInstance <- sc3
11587-
# call frame return <- mintRetSrc
11588-
# call frame return
1158911572
# call frame
1159011573
# call frame
1159111574
# call frame
11592-
# call frame <- sp
11575+
# call frame <- callee's SP (not yet restored)
1159311576

1159411577
# note: we don't care about t3 anymore
1159511578
if ARM64 or ARM64E

‎Source/JavaScriptCore/offlineasm/parser.rb‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -955,6 +955,9 @@ def parseIncludes(final, comment, options)
955955
def readTextFile(fileName)
956956
data = IO::read(fileName)
957957

958+
# https://commits.webkit.org/310937@main broke builds because the files contained non-ASCII characters. Let's forbid non-ASCII early so we don't hit problems downstream.
959+
raise "#{fileName} contains non-ASCII characters" unless data.ascii_only?
960+
958961
# On Windows, files may contain CRLF line endings (for example, git client might
959962
# automatically replace \n with \r\n on Windows) which will fail our parsing.
960963
# Thus, we'll just remove all \r from the data (keeping just the \n characters)

‎Source/JavaScriptCore/wasm/WasmBBQJIT.cpp‎

Lines changed: 10 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4344,20 +4344,6 @@ void BBQJIT::returnValuesFromCall(Vector<Value, N>& results, const FunctionSigna
43444344
ASSERT(!currentBinding.isScratch());
43454345
} else {
43464346
ASSERT(returnLocation.isStackArgument());
4347-
// FIXME: Ideally, we would leave these values where they are but a subsequent call could clobber them before they are used.
4348-
// That said, stack results are very rare so this isn't too painful.
4349-
// Even if we did leave them where they are, we'd need to flush them to their canonical location at the next branch otherwise
4350-
// we could have something like (assume no result regs for simplicity):
4351-
// call (result i32 i32) $foo
4352-
// if (result i32) // Stack: i32(StackArgument:8) i32(StackArgument:0)
4353-
// // Stack: i32(StackArgument:8)
4354-
// else
4355-
// call (result i32 i32) $bar // Stack: i32(StackArgument:8) we have to flush the stack argument to make room for the result of bar
4356-
// drop // Stack: i32(Stack:X) i32(StackArgument:8) i32(StackArgument:0)
4357-
// drop // Stack: i32(Stack:X) i32(StackArgument:8)
4358-
// end
4359-
// return // Stack i32(*Conflicting locations*)
4360-
43614347
Location canonicalLocation = canonicalSlot(result);
43624348
emitMoveMemory(result.type(), returnLocation, canonicalLocation);
43634349
returnLocation = canonicalLocation;
@@ -4521,7 +4507,11 @@ void BBQJIT::emitTailCall(FunctionSpaceIndex functionIndexSpace, const TypeDefin
45214507
});
45224508
}
45234509

4524-
// Our callee could have tail called someone else and changed SP so we need to restore it. Do this before restoring our results since results are stored at the top of the reserved stack space.
4510+
// Push return value(s) onto the expression stack. Read results before restoring SP
4511+
// since results are at the bottom of the arg/result area, addressable from the callee's SP.
4512+
returnValuesFromCall(results, functionType, callInfo);
4513+
4514+
// Our callee could have tail called someone else and changed SP so we need to restore it.
45254515
m_frameSizeLabels.append(m_jit.moveWithPatch(TrustedImmPtr(nullptr), wasmScratchGPR));
45264516
#if CPU(ARM64)
45274517
m_jit.subPtr(GPRInfo::callFrameRegister, wasmScratchGPR, MacroAssembler::stackPointerRegister);
@@ -4530,9 +4520,6 @@ void BBQJIT::emitTailCall(FunctionSpaceIndex functionIndexSpace, const TypeDefin
45304520
m_jit.move(wasmScratchGPR, MacroAssembler::stackPointerRegister);
45314521
#endif
45324522

4533-
// Push return value(s) onto the expression stack
4534-
returnValuesFromCall(results, functionType, callInfo);
4535-
45364523
if (m_info.callCanClobberInstance(functionIndexSpace) || m_info.isImportedFunctionFromFunctionIndexSpace(functionIndexSpace))
45374524
restoreWebAssemblyGlobalStateAfterWasmCall();
45384525

@@ -4604,8 +4591,12 @@ void BBQJIT::emitIndirectCall(const char* opcode, unsigned callProfileIndex, con
46044591
m_jit.loadPtr(CCallHelpers::Address(importableFunction, WasmToWasmImportableFunction::offsetOfEntrypointLoadLocation()), wasmScratchGPR);
46054592
m_jit.call(CCallHelpers::Address(wasmScratchGPR), WasmEntryPtrTag);
46064593

4607-
// Our callee could have tail called someone else and changed SP so we need to restore it. Do this before restoring our results since results are stored at the top of the reserved stack space.
4594+
// Read results before restoring SP since results are at the bottom of the
4595+
// arg/result area, addressable from the callee's SP.
46084596
afterCall.link(m_jit);
4597+
returnValuesFromCall(results, *signature.as<FunctionSignature>(), wasmCalleeInfo);
4598+
4599+
// Our callee could have tail called someone else and changed SP so we need to restore it.
46094600
m_frameSizeLabels.append(m_jit.moveWithPatch(TrustedImmPtr(nullptr), wasmScratchGPR));
46104601
#if CPU(ARM64)
46114602
m_jit.subPtr(GPRInfo::callFrameRegister, wasmScratchGPR, MacroAssembler::stackPointerRegister);
@@ -4614,8 +4605,6 @@ void BBQJIT::emitIndirectCall(const char* opcode, unsigned callProfileIndex, con
46144605
m_jit.move(wasmScratchGPR, MacroAssembler::stackPointerRegister);
46154606
#endif
46164607

4617-
returnValuesFromCall(results, *signature.as<FunctionSignature>(), wasmCalleeInfo);
4618-
46194608
restoreWebAssemblyGlobalStateAfterWasmCall();
46204609

46214610
LOG_INSTRUCTION(opcode, callee, arguments, "=> ", results);

‎Source/JavaScriptCore/wasm/WasmCallingConvention.h‎

Lines changed: 7 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -187,57 +187,6 @@ class WasmCallingConvention {
187187
RELEASE_ASSERT_NOT_REACHED();
188188
}
189189

190-
uint32_t numberOfStackResults(const FunctionSignature& signature) const
191-
{
192-
const uint32_t gprCount = jsrArgs.size();
193-
const uint32_t fprCount = fprArgs.size();
194-
uint32_t gprIndex = 0;
195-
uint32_t fprIndex = 0;
196-
uint32_t stackCount = 0;
197-
for (uint32_t i = 0; i < signature.returnCount(); i++) {
198-
switch (signature.returnType(i).kind) {
199-
case TypeKind::I32:
200-
case TypeKind::I64:
201-
case TypeKind::Exnref:
202-
case TypeKind::Externref:
203-
case TypeKind::Funcref:
204-
case TypeKind::RefNull:
205-
case TypeKind::Ref:
206-
if (gprIndex < gprCount)
207-
++gprIndex;
208-
else
209-
++stackCount;
210-
break;
211-
case TypeKind::F32:
212-
case TypeKind::F64:
213-
case TypeKind::V128:
214-
if (fprIndex < fprCount)
215-
++fprIndex;
216-
else
217-
++stackCount;
218-
break;
219-
case TypeKind::Void:
220-
case TypeKind::Func:
221-
case TypeKind::Struct:
222-
case TypeKind::Structref:
223-
case TypeKind::Array:
224-
case TypeKind::Arrayref:
225-
case TypeKind::Eqref:
226-
case TypeKind::Anyref:
227-
case TypeKind::Noexnref:
228-
case TypeKind::Noneref:
229-
case TypeKind::Nofuncref:
230-
case TypeKind::Noexternref:
231-
case TypeKind::I31ref:
232-
case TypeKind::Sub:
233-
case TypeKind::Subfinal:
234-
case TypeKind::Rec:
235-
RELEASE_ASSERT_NOT_REACHED();
236-
}
237-
}
238-
return stackCount;
239-
}
240-
241190
public:
242191

243192
CallInformation callInformationFor(const TypeDefinition& type, CallRole role = CallRole::Caller) const
@@ -262,22 +211,17 @@ class WasmCallingConvention {
262211
[&](unsigned index) {
263212
return marshallLocation(role, signature.argumentType(index), gpArgumentCount, fpArgumentCount, argStackOffset);
264213
});
265-
uint32_t stackArgsInBytes = argStackOffset - headerSize;
266214

267215
gpArgumentCount = 0;
268216
fpArgumentCount = 0;
269-
size_t stackResults = numberOfStackResults(signature);
270-
// N.B. this is inaccurate for vector results. In that case and when the actual result space is larger than the argument space, there is a quirk in
271-
// the calling convention where the argument and result space is not minimal, i.e. arguments and results don't overlap as much as they could.
272-
uint32_t estimatedStackResultsInBytes = stackResults * sizeof(Register);
273-
uint32_t estimatedTotalArgAndResultsInBytes = WTF::roundUpToMultipleOf<stackAlignmentBytes()>(std::max(stackArgsInBytes, estimatedStackResultsInBytes));
274-
size_t resultStackOffset = headerSize + estimatedTotalArgAndResultsInBytes - estimatedStackResultsInBytes;
217+
size_t resultStackOffset = headerSize;
275218
Vector<ArgumentLocation, 1> results(signature.returnCount(),
276219
[&](unsigned index) {
277220
return marshallLocation(role, signature.returnType(index), gpArgumentCount, fpArgumentCount, resultStackOffset);
278221
});
279-
size_t totalFrameSize = resultStackOffset;
280-
ASSERT(totalFrameSize >= argStackOffset);
222+
223+
ASSERT(!(headerSize % stackAlignmentBytes()));
224+
size_t totalFrameSize = WTF::roundUpToMultipleOf<stackAlignmentBytes()>(std::max(argStackOffset, resultStackOffset));
281225

282226
return { thisArgument, WTF::move(params), WTF::move(results), totalFrameSize, headerSize };
283227
}
@@ -550,21 +494,17 @@ class CCallingConventionArmThumb2 {
550494
ASSERT(!argumentType.isV128());
551495
return marshallLocation(role, argumentType, gpArgumentCount, fpArgumentCount, argStackOffset);
552496
});
553-
uint32_t stackArgs = argStackOffset - headerSize;
554-
size_t stackArgsCount = numberOfStackArguments(signature);
555497

556498
gpArgumentCount = 0;
557499
fpArgumentCount = 0;
558-
size_t stackResultsCount = numberOfStackResults(signature);
559-
uint32_t stackResults = stackResultsCount * sizeof(Register);
560-
uint32_t stackCountAligned = WTF::roundUpToMultipleOf<stackAlignmentBytes()>(std::max(stackArgs, stackResults));
561-
size_t resultStackOffset = headerSize + stackCountAligned - stackResults;
500+
size_t resultStackOffset = headerSize;
562501
Vector<ArgumentLocation, 1> results(signature.returnCount(),
563502
[&](unsigned index) {
564503
ASSERT(!signature.returnType(index).isV128());
565504
return marshallLocation(role, signature.returnType(index), gpArgumentCount, fpArgumentCount, resultStackOffset);
566505
});
567-
return { thisArgument, WTF::move(params), WTF::move(results), std::max(argStackOffset, resultStackOffset), std::max(stackArgsCount, stackResultsCount) };
506+
size_t totalFrameSize = headerSize + WTF::roundUpToMultipleOf<stackAlignmentBytes()>(std::max(argStackOffset - headerSize, resultStackOffset - headerSize));
507+
return { thisArgument, WTF::move(params), WTF::move(results), totalFrameSize, headerSize };
568508
}
569509

570510
const Vector<GPRReg> gprArgs;

0 commit comments

Comments
 (0)