Skip to content

test(RemoteConfig): keep late callbacks from crashing APITests - #16783

Open
paulb777 wants to merge 1 commit into
mainfrom
pb-rc-visionos-apitests
Open

paulb777 wants to merge 1 commit into
mainfrom
pb-rc-visionos-apitests

Conversation

@paulb777

@paulb777 paulb777 commented Oct 2, 2026

Copy link
Copy Markdown
Member

Problem

The RemoteConfigFakeConsole visionOS jobs (spm_2) are flaky:

The same sequence happens each time:

  1. APITests testFetchThenActivate, testFetchWithExpirationThenActivate and testUnchangedActivateWillFlag exceed their 10 s expectation timeout.
  2. Their activate completions arrive later, after tearDown() has set config = nil. They then crash the test process with APITests.swift:46: Fatal error: Unexpectedly found nil while implicitly unwrapping an Optional value (and the same at :78).
  3. xcodebuild restarts the test runner. On Xcode 27, the recorded failures also trigger the slow diagnostics collection that fix(ci): skip test diagnostics collection in SPM test runs #16781 skips.

Change

APITests and FakeConsoleTests now capture a local strong reference to config at the start of each test and use it in the completion handlers. A late handler runs against a valid instance instead of crashing. The pattern is documented on APITestBase.config.

Testing

I reproduced the CI crash locally on a visionOS 26.5 simulator. I saturated the host CPU with 24 busy loops on a 16-core Mac and temporarily cut the APITests timeout to 0.3 s.

Run Result
Before this change APITests.swift:46, :60 and :78 crash with the CI fatal error; the runner restarts twice
After this change The same 3 timeouts are reported as ordinary failures; no crash, no restart, no misattributed failures; the other 45 tests run normally
After, normal 10 s timeout, no load 48/48 RemoteConfigFakeConsole tests pass

Why the completions are late

This PR stops the crash. It does not stop the timeouts.

activate's completion waits for -[FIRExperimentController updateExperimentsWithServiceOrigin:...], which dispatches to dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0). When the CPU is saturated, background-QoS work is starved. CI runners hosting the visionOS simulator are in exactly that state.

Under the same local load, I changed only that dispatch's QoS:

ABTesting dispatch QoS APITests total time
background (current) 6.2 s, 9.1 s, 10.7 s
utility 0.30 s, 0.18 s
default 0.18 s, 0.26 s, 0.21 s

Without load, APITests takes about 0.1 s on the current code. Fixing the delay needs an A/B Testing SDK change, so it's out of scope for this test-only PR.

#no-changelog

APITests and FakeConsoleTests read `self.config` inside fetch and
activate completion handlers, but APITestBase.tearDown() sets `config`
to nil. When a test times out, its activate completion can still run
later, unwrap nil, and crash the test process. On the visionOS CI jobs
this turned three expectation timeouts into a crash and a test-runner
restart.

Capture a local strong reference to `config` at the start of each test
and use it in the handlers, so a late handler runs against a valid
instance instead of crashing. Document the pattern on
APITestBase.config.
@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant