Skip to content

"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

Closed

Description

@ned-deily

As of Xcode 16, Apple has changed the command interface to xcresultool such that the way it is invoked when running the testios recipe in Makefile.pre.in is no longer valid, causing the iOS test step to fail while attempting to extract the results of the test run.

[...]
# Regardless of success or failure, extract and print the test output
xcrun xcresulttool get --path iOSTestbed.arm64-iphonesimulator.1731824398/arm64-iphonesimulator.xcresult \
		--id $( xcrun xcresulttool get --path iOSTestbed.arm64-iphonesimulator.1731824398/arm64-iphonesimulator.xcresult --format json | _PYTHON_PROJECT_BASE=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator _PYTHON_HOST_PLATFORM=ios-13.0-arm64-iphonesimulator PYTHONPATH=../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__ios_arm64-iphonesimulator _PYTHON_SYSCONFIGDATA_PATH=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator/build/lib.ios-13.0-arm64-iphonesimulator-3.14 /Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-darwin/root-arm64-apple-darwin/bin/python3 -c "import sys, json; result = json.load(sys.stdin); print(result['actions']['_values'][0]['actionResult']['logRef']['id']['_value'])" ) \
		--format json | \
		_PYTHON_PROJECT_BASE=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator _PYTHON_HOST_PLATFORM=ios-13.0-arm64-iphonesimulator PYTHONPATH=../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__ios_arm64-iphonesimulator _PYTHON_SYSCONFIGDATA_PATH=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator/build/lib.ios-13.0-arm64-iphonesimulator-3.14 /Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-darwin/root-arm64-apple-darwin/bin/python3 -c "import sys, json; result = json.load(sys.stdin); print(result['subsections']['_values'][1]['subsections']['_values'][0]['emittedOutput']['_value'])"
Error: This command is deprecated and will be removed in a future release, --legacy flag is required to use it.
Usage: xcresulttool get object [--legacy] --path <path> [--id <id>] [--version <version>] [--format <format>]
  See 'xcresulttool get object --help' for more information.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sys, json; result = json.load(sys.stdin); print(result['actions']['_values'][0]['actionResult']['logRef']['id']['_value'])
                               ~~~~~~~~~^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 293, in load
    return loads(fp.read(),
        cls=cls, object_hook=object_hook,
        parse_float=parse_float, parse_int=parse_int,
        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: Missing value for '--id <id>'
Help:  --id <id>  The ID of the object [optional, assumes rootID if not specified].
Usage: xcresulttool get object [--legacy] --path <path> [--id <id>] [--version <version>] [--format <format>]
  See 'xcresulttool get object --help' for more information.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sys, json; result = json.load(sys.stdin); print(result['subsections']['_values'][1]['subsections']['_values'][0]['emittedOutput']['_value'])
                               ~~~~~~~~~^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 293, in load
    return loads(fp.read(),
        cls=cls, object_hook=object_hook,
        parse_float=parse_float, parse_int=parse_int,
        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
make: *** [testios] Error 1

Linked PRs

Activity

  1. added
    3.13only security fixes
    3.14bugs and security fixes
    and removed on Nov 17, 2024
  2. freakboy3742 commented on Nov 18, 2024

    @freakboy3742
    Contributor

    See #126821 for related issue/feature. Adding stdout/stderr redirection will enable real-time log streaming, which means we don't need to extract stdout/err from the XCresult file, and will also allow progress tracking of a running test suite.

  3. added
    testsTests in the Lib/test dir
    buildThe build process and cross-build
    on Dec 5, 2024
  4. added a commit that references this issue on Dec 9, 2024
  5. added 4 commits that reference this issue on Dec 9, 2024
  6. 1 remaining item

  7. added 3 commits that reference this issue on Dec 13, 2024
  8. added a commit that references this issue on Jan 8, 2025
  9. added 3 commits that reference this issue on Jan 21, 2025
  10. added 3 commits that reference this issue on Feb 3, 2026
  11. added 2 commits that reference this issue on Jun 12, 2026
  12. added 2 commits that reference this issue on Oct 2, 2026
  13. added a commit that references this issue on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.13only security fixes3.14bugs and security fixesOS-iosbuildThe build process and cross-buildtestsTests in the Lib/test dir

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions