Skip to content

iOS test simulators fail to start on 20250811 macOS-15 build #12777

Open

Description

@freakboy3742

Description

As of the 20250811 macos-15-arm64 release, Xcode appears to be unable to launch iOS simulators as part of an XCUnit test suite.

I have a XCUnit test suite that has worked for some time. It runs happily using Xcode 16.4 and an iOS 18.6 simulator on my own machine. It has also run successfully on every other iOS micro versions. It also ran on Xcode 16.0 and iOS 18.0 on the previous macos-15-arm64 image.

I am aware of #12541 and the changes to the included iOS SDK runtimes. I have applied the recommended change to explicitly select Xcode 16.4 in my actions workflow.

However, as of the macOS image change change, when I run my test suite, I get errors indicating that Xcode is unable to start an iOS simulator.

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 22.04
  • Ubuntu 24.04
  • macOS 13
  • macOS 13 Arm64
  • macOS 14
  • macOS 14 Arm64
  • macOS 15
  • macOS 15 Arm64
  • Windows Server 2019
  • Windows Server 2022
  • Windows Server 2025

Image version and build link

The problem has been observed in:

Is it regression?

Yes; any build against the previous macOS-15-arm64 image.

Expected behavior

It should be possible to run an Xcode project that starts an iOS simulator using:

xcodebuild test -project localtest/iOSTestbed.xcodeproj -scheme iOSTestbed -destination 'platform=iOS Simulator,name=iPhone 16e'

Actual behavior

The test project builds, but then fails to start the simulator, with an error:

  Writing result bundle at path:
  	/private/var/folders/x7/ch5v91h56_zbvbd1y2f600dm0000gn/T/cibw-run-lfch6xkz/cp313-ios_arm64_iphonesimulator/testbed/20250818-071548.xcresult
  2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions Finished with error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices
  Domain: com.apple.CoreSimulator.SimError
  Code: 404
  --
  2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions com.apple.dt.IDERunOperationWorkerFinished {
      "operation_duration_ms" = 8;
      "operation_errorCode" = 404;
      "operation_errorDomain" = "com.apple.CoreSimulator.SimError";
      "operation_name" = "_IDEInstalliPhoneSimulatorWorker";
  }
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 elapsed -- Testing started completed.
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 sec, +9.957 sec -- end
  Testing failed:
  	iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices))

If I ssh into the machine and run the suite manually, I get the following error:

Writing result bundle at path:
        /Users/runner/work/pyspamsum/pyspamsum/localtest/20250818-055037.xcresult

2025-08-18 05:50:49.068 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: Finished with error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices
Domain: com.apple.CoreSimulator.SimError
Code: 404
User Info: {
    DVTErrorCreationDateKey = "2025-08-18 05:50:49 +0000";
    IDERunOperationFailingWorker = "_IDEInstalliPhoneSimulatorWorker";
}
--

2025-08-18 05:50:49.069 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: com.apple.dt.IDERunOperationWorkerFinished {
    "device_identifier" = "FC1D56A6-910E-4DDA-8DC7-776480505F9C";
    "device_model" = "iPhone17,5";
    "device_osBuild" = "26.0 (23A5308g)";
    "device_platform" = "com.apple.platform.iphonesimulator";
    "device_thinningType" = "iPhone17,5";
    "dvt_coredevice_version" = "477.29";
    "dvt_coresimulator_version" = 1047;
    "dvt_mobiledevice_version" = "1818.0.1";
    "launchSession_schemeCommand" = Test;
    "launchSession_state" = 1;
    "launchSession_targetArch" = arm64;
    "operation_duration_ms" = 11;
    "operation_errorCode" = 404;
    "operation_errorDomain" = "com.apple.CoreSimulator.SimError";
    "operation_errorWorker" = "_IDEInstalliPhoneSimulatorWorker";
    "operation_name" = IDERunOperationWorkerGroup;
    "param_debugger_attachToExtensions" = 0;
    "param_debugger_attachToXPC" = 0;
    "param_debugger_type" = 1;
    "param_destination_isProxy" = 0;
    "param_destination_platform" = "com.apple.platform.iphonesimulator";
    "param_diag_113575882_enable" = 0;
    "param_diag_MainThreadChecker_stopOnIssue" = 0;
    "param_diag_MallocStackLogging_enableDuringAttach" = 0;
    "param_diag_MallocStackLogging_enableForXPC" = 0;
    "param_diag_allowLocationSimulation" = 1;
    "param_diag_checker_tpc_enable" = 0;
    "param_diag_gpu_frameCapture_enable" = 3;
    "param_diag_gpu_shaderValidation_enable" = 0;
    "param_diag_gpu_validation_enable" = 1;
    "param_diag_guardMalloc_enable" = 0;
    "param_diag_memoryGraphOnResourceException" = 0;
    "param_diag_mtc_enable" = 0;
    "param_diag_queueDebugging_enable" = 1;
    "param_diag_runtimeProfile_generate" = 1;
    "param_diag_sanitizer_asan_enable" = 0;
    "param_diag_sanitizer_tsan_enable" = 0;
    "param_diag_sanitizer_tsan_stopOnIssue" = 0;
    "param_diag_sanitizer_ubsan_enable" = 0;
    "param_diag_sanitizer_ubsan_stopOnIssue" = 0;
    "param_diag_showNonLocalizedStrings" = 0;
    "param_diag_viewDebugging_enabled" = 0;
    "param_diag_viewDebugging_insertDylibOnLaunch" = 0;
    "param_install_style" = 2;
    "param_launcher_UID" = 2;
    "param_launcher_allowDeviceSensorReplayData" = 0;
    "param_launcher_kind" = 0;
    "param_launcher_style" = 0;
    "param_launcher_substyle" = 0;
    "param_runnable_appExtensionHostRunMode" = 0;
    "param_runnable_productType" = "com.apple.product-type.application";
    "param_structuredConsoleMode" = 0;
    "param_testing_launchedForTesting" = 1;
    "param_testing_suppressSimulatorApp" = 1;
    "param_testing_usingCLI" = 0;
    "sdk_canonicalName" = "iphonesimulator18.5";
    "sdk_osVersion" = "18.5";
    "sdk_variant" = iphonesimulator;
}
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 elapsed -- Testing started completed.
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 sec, +12.606 sec -- end
Testing failed:
        iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices))

The only thing that stands out in this error log is the reference to Xcode 26; given than I'm explicitly using Xcode 16.4, it's not clear to me why iOS 26 would be involved.

Repro steps

The test suites that are affected are all using CPython's iOS testbed, by way of cibuildwheel.

The problem can be reproduced by downloading this tarball, unpacking it, then running:

$ curl -L https://github.com/beeware/Python-Apple-support/releases/download/3.13-b10/Python-3.13-iOS-support.b10.tar.gz -o Python-3.13-iOS-support.b10.tar.gz
$ tar zxvf Python-3.13-iOS-support.b10.tar.gz
$ python3.13 testbed clone localtest
$ python3.13 localtest run -- test test_platform

This will run a minimal subset of the Python test suite in an iPhone 16e simulator.

This sequence of commands works locally, but fails in the macos-15-arm64 environment as of the 20250811 update, regardless of what Xcode version has been set as the default.

localtest/__main__.py contains the code to launch Xcode and stream logs from the simulator that is created. The code starting xcodebuild is around L248 of that script.

Activity

  1. badredaha commented on Aug 18, 2025

    @badredaha

    Hi,

    same error here +1

  2. archita105 commented on Aug 18, 2025

    @archita105
    Contributor

    Hi @freakboy3742 - Thank you for bringing this issue to our attention. We will look into this issue and will update you .

  3. gruselhaus commented on Aug 18, 2025

    @gruselhaus

    @archita105 this is directly connected to #12768.

    This completely broke our CI.

  4. erik-bershel commented on Aug 18, 2025

    @erik-bershel
    Contributor

    Hey @gruselhaus!

    The mentioned PR cannot affect the problem mentioned by the topic starter, since it belongs to a different version of the image (Intel-based macOS-15) and the changes have not yet been delivered.

  5. erik-bershel commented on Aug 18, 2025

    @erik-bershel
    Contributor

    Hey @freakboy3742!

    The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling xcodebuild, judging by the general log. If the test suite recognizes Xcode 26 components as belonging to Xcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.

  6. gruselhaus commented on Aug 18, 2025

    @gruselhaus

    Hey @gruselhaus!

    The mentioned PR cannot affect the problem mentioned by the topic starter, since it belongs to a different version of the image (Intel-based macOS-15) and the changes have not yet been delivered.

    Yeah, sorry. You're absolutely right.

  7. gruselhaus commented on Aug 18, 2025

    @gruselhaus

    Hey @freakboy3742!

    The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling xcodebuild, judging by the general log. If the test suite recognizes Xcode 26 components as belonging to Xcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.

    We've added a work around to force a Xcode version

          # -------------------------
          # Setup Xcode (Hotfix, see: https://github.com/actions/runner-images/issues/12777)
          # -------------------------
          - name: Use Xcode 16.4
            uses: maxim-lobanov/setup-xcode@v1
            with:
              xcode-version: "16.4"
    
          - name: Download iOS 18 platform
            run: |
              sudo xcodebuild -license accept || true
              sudo xcodebuild -downloadPlatform iOS
              xcrun simctl list runtimes
  8. 37 remaining items

  9. doulos76 commented on Jun 3, 2026

    @doulos76

    Adding a data point with a narrowed-down root cause — hope it helps.

    Same symptom, narrowed to ExtensionFoundation._EXHostConfiguration

    Our iOS unit-test workflow fails on macos-15 with xctest <external symbol> / SIGBUS exit 65 — even with the "stable" combo currently recommended in #13264 (Xcode 16.4 + iOS 18.6 + iPhone 16). Local runs on macOS 26 (iOS 26.1 simulator) pass. We collected the crashlog and traced it down to a missing external symbol that Apple's ExtensionFoundation framework reaches from the simulator.

    Crash report

    exception : EXC_BAD_ACCESS / SIGBUS
    codes     : 0x0000000000000101, 0x000000000bad4007   ← dyld missing-symbol sentinel
    host      : macOS 15.7.7 (24G720)  ← GitHub Actions VM image, constant across all our tries
    proc      : /Applications/Xcode_16.4.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Xcode/Agents/xctest
    

    Faulting thread (bottom-up):

    +[XCTestCase _allSubclasses]              (XCTestCore)
      → objc_copyClassList                    (libobjc)
      → realizeAllClasses                     (libobjc)
      → realizeClassMaybeSwiftMaybeRelock     (libobjc)
      → type metadata accessor for _EXHostConfiguration   (ExtensionFoundation, in iOS simruntime)
      → swift_getSingletonMetadata            (libswiftCore)
      → swift::MetadataCacheEntryBase::doInitialization
      → type metadata completion for _EXHostConfiguration (ExtensionFoundation)
      → [image: name=None, path=None, base=0x0]   ← dyld 0xBAD4007 stub for a symbol the host macOS doesn't export
      → SIGBUS @ 0xBAD4007
    

    So the iOS simruntime's ExtensionFoundation.framework references an external symbol that doesn't exist on the host macOS 15.7.7 ExtensionFoundation/Foundation. dyld stubs the unresolved symbol with 0xBAD4007. Then xctest's bootstrap path (+[XCTestCase _allSubclasses] → objc_copyClassList → realizeAllClasses) eagerly realizes every class — including _EXHostConfiguration, whose metadata completion function calls the stubbed symbol — and dies immediately. This explains why narrowing to a single, completely empty test bundle (-only-testing:CommonTests with no Extension code) fails at the same place: the crash happens before any test runs, during ObjC class realization.

    Our failure matrix (6/6, all the same crash pattern)

    Runner Xcode iOS sim Device Result
    macos-15 16.4 18.5 iPhone 16 Pro SIGBUS
    macos-14 16.1 17.5 (sdk 18.1) SEGV
    macos-14 16.1 18.1 iPhone 15 SIGBUS
    macos-14 15.4 17.5 iPhone 15 SEGV <external symbol>
    macos-15 26.1.1 26.1 iPhone 17 Pro SIGBUS <external symbol> (crashlog confirmed)
    macos-15 16.4 18.6 iPhone 16 SIGBUS — same _EXHostConfiguration stub (crashlog confirmed)

    Host macOS 15.7.7 is constant across every run. The crash location and the missing symbol are identical between the Xcode 26.1.1 / iOS 26.1 and the Xcode 16.4 / iOS 18.6 attempts, so it isn't sensitive to the destination axis — it's that host-side ExtensionFoundation is missing whatever symbol any of these simruntimes import.

    Reproduction artifacts

    • Workflow used: empty single-test bundle, -only-testing:CommonTests, CODE_SIGNING_ALLOWED=NO — minimal repro.
    • We collected ~/Library/Logs/DiagnosticReports/xctest-*.ips via an if: always() step and uploaded as a crashlogs artifact. Anyone hitting this can do the same to confirm whether their crash points to _EXHostConfiguration too.

    Hope the narrowed-down stack and the missing-symbol identity helps with whatever fix lands on the runner-images side.

  10. freakboy3742 commented on Jun 3, 2026

    @freakboy3742
    Author

    @erik-bershel Any update on this issue from GitHub's side would be very much appreciated. As it stands, macOS-14 is the only viable iOS simulator platform that GitHub offers at present, and the macOS-14 deprecation cycle is a month away. If GitHub is intending to proceed with that deprecation without fixing this issue (or having a workaround), that's going to cause a major disruption to anyone trying to use GitHub actions for iOS testing.

  11. freakboy3742 commented on Jun 4, 2026

    @freakboy3742
    Author

    Macos 14 runner is not deleted until november

    I am aware. But the deprecation process begins on July 6, and the deprecation process includes brownouts in runner availability over the deprecation period - and I would like to have a fix in place significantly before November.

    EDIT: This was a response to a comment from user pierreluc16, which appears to have been memory holed without a trace.

  12. freakboy3742 commented on Jul 13, 2026

    @freakboy3742
    Author

    I believe I may have just stumbled on the cause of the problem here.

    There's actually 2 problems:

    1. iOS test simulators failing to start, or having exceptionally poor performance.
    2. xcodebuild reporting that there are no simulators available.

    (1) was the subject of this original report in August 2025, related to the 20250811 macOS images. I can't speak for the experience of anyone else, but I haven't seen this problem since late 2025. There are some comments on this thread that suggest it may have been related to a performance issue with macOS 26, resolved by Apple around November / macOS 26.2 that resolved that problem.

    (2) was originally reported separately as #13228, but that was closed as a duplicate of this report. However, as a result of an accidental discovery yesterday, I believe I can confirm it was a completely separate issue, introduced by a change in Xcode 16 (or, at least, a change in performance on the macos-15+ runners that causes a race condition that didn't previously exist).

    When using Xcode 16+, running xcodebuild to start an iOS app doesn't wait until the CoreSimulatorService is actually running before trying to start the app, instead reporting that there are no simulators running if the simulator service isn't available. You can verify this by obtaining an iOS Xcode project, and running

    sudo pkill -f CoreSimulatorService; xcodebuild -showdestinations -project path/to/project.xcodeproj
    

    You may also need a -scheme argument for xcodebuild, depending on the project you use.

    In my testing on an M5 MacBook, this will, about 2 in 3 attempts, report "no simulators available" - which was the behavior reported by #13228, and I have been observing in 1 in 10 runs on the macos-15 and macos-26 runners.

    However, it appears that simctl does block on the availability of the CoreSimulatorService. If you do the same test with xcrun simctl:

    sudo pkill -f CoreSimulatorService; xcrun simctl list devices
    

    The call to simctl may take a couple of seconds while the simulator service restarts, but it will always report the full list of simulators - never a "no simulators" result.

    So - the workaround appears to be invoking simctl, even if the result is ignored, prior to running xcodebuild. This ensures that the daemon is running prior to trying to start the build.

    I can't easily verify if this was also a problem on Xcode 15+, or if the macos-14 runners are just slow enough that the CoreSimulatorService was always started in time. But, introducing this workaround, I'm no longer seeing the instability in availability of simulators in xcodebuild runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions