xctestrunner incorrectly handles maxRuntimeVersion (15.255.255 → 15.3) #72
Description
Summary
xctestrunner incorrectly parses the maxRuntimeVersion of iPhone 6s Plus (and similar devices) as 15.3, which makes it impossible to create a simulator with iOS 15.5.
Steps to Reproduce
Run following command on the valid bazel project:
bazel test --ios_simulator_device='iPhone 6s Plus' --ios_simulator_version=15.5Actual Behavior
The test runner fails with:
Traceback (most recent call last):
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/_main/../rules_apple++non_module_deps+xctestrunner/xctestrunner/test_runner/ios_test_runner.py", line 319, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/_main/../rules_apple++non_module_deps+xctestrunner/xctestrunner/test_runner/ios_test_runner.py", line 313, in main
exit_code = args.func(args)
^^^^^^^^^^^^^^^
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/_main/../rules_apple++non_module_deps+xctestrunner/xctestrunner/test_runner/ios_test_runner.py", line 216, in _SimulatorTest
return _RunSimulatorTest(args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/_main/../rules_apple++non_module_deps+xctestrunner/xctestrunner/test_runner/ios_test_runner.py", line 186, in _RunSimulatorTest
simulator_id, _, os_version, _ = simulator_util.CreateNewSimulator(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/rules_apple++non_module_deps+xctestrunner/xctestrunner/simulator_control/simulator_util.py", line 358, in CreateNewSimulator
_ValidateSimulatorTypeWithOsVersion(device_type, os_version)
File "/path/to/bazel//a59b5fbd1ac4fd988fa8e6c30761d359/sandbox/darwin-sandbox/487/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/Tests/Foundation/NSData_zlibTests/NSData_zlibUnitTest.runfiles/rules_apple++non_module_deps+xctestrunner/xctestrunner/simulator_control/simulator_util.py", line 654, in _ValidateSimulatorTypeWithOsVersion
raise ios_errors.IllegalArgumentError(
xctestrunner.shared.ios_errors.IllegalArgumentError: The max OS version of iPhone 6s Plus is 15.300000. But current OS version is 15.5
error: tests exited with '1
Root Cause
This happens because xctestrunner/simulator_control/simtype_profile.py does not handle special maxRuntimeVersion patterns correctly. On my machine:
$ xcrun simctl list devicetypes -j | python3 -c "
import json, sys
data = json.load(sys.stdin)
iphone_devices = [d for d in data['devicetypes'] if d['name'].startswith('iPhone')]
print('iPhone Runtime Version Analysis:')
print('=' * 50)
for device in iphone_devices:
max_runtime = device.get('maxRuntimeVersionString', 'unlimited')
print(f'{device[\"name\"]:30} Max Runtime: {max_runtime}')
"shows:
iPhone Runtime Version Analysis:
==================================================
iPhone 16 Pro Max Runtime: 65535.255.255
iPhone 16 Pro Max Max Runtime: 65535.255.255
iPhone 16e Max Runtime: 65535.255.255
iPhone 16 Max Runtime: 65535.255.255
iPhone 16 Plus Max Runtime: 65535.255.255
iPhone 15 Pro Max Runtime: 65535.255.255
iPhone 15 Pro Max Max Runtime: 65535.255.255
iPhone 15 Max Runtime: 65535.255.255
iPhone 15 Plus Max Runtime: 65535.255.255
iPhone 14 Pro Max Runtime: 65535.255.255
iPhone 14 Pro Max Max Runtime: 65535.255.255
iPhone 14 Max Runtime: 65535.255.255
iPhone 14 Plus Max Runtime: 65535.255.255
iPhone SE (3rd generation) Max Runtime: 65535.255.255
iPhone 13 Pro Max Runtime: 65535.255.255
iPhone 13 Pro Max Max Runtime: 65535.255.255
iPhone 13 Max Runtime: 65535.255.255
iPhone 13 mini Max Runtime: 65535.255.255
iPhone 12 Pro Max Runtime: 65535.255.255
iPhone 12 Pro Max Max Runtime: 65535.255.255
iPhone 12 Max Runtime: 65535.255.255
iPhone 12 mini Max Runtime: 65535.255.255
iPhone SE (2nd generation) Max Runtime: 65535.255.255
iPhone 11 Pro Max Runtime: 65535.255.255
iPhone 11 Pro Max Max Runtime: 65535.255.255
iPhone 11 Max Runtime: 65535.255.255
iPhone Xʀ Max Runtime: 18.99.0
iPhone Xs Max Runtime: 18.99.0
iPhone Xs Max Max Runtime: 18.99.0
iPhone X Max Runtime: 16.9.0
iPhone 8 Max Runtime: 16.9.0
iPhone 8 Plus Max Runtime: 16.9.0
iPhone 7 Max Runtime: 15.255.255
iPhone 7 Plus Max Runtime: 15.255.255
iPhone SE (1st generation) Max Runtime: 15.255.255
iPhone 6s Max Runtime: 15.255.255
iPhone 6s Plus Max Runtime: 15.255.255
Considering the model's last supported iOS versions, the following patterns are used:
65535.255.255→ effectively unlimited18.99.0→ upper bound pattern15.255.255→ upper bound pattern
However, xctestrunner truncates these values incorrectly, causing 15.255.255 to be parsed as 15.3.
Expected Behavior
iPhone 6s Plusshould be treated as supporting up to iOS 15.x, so requesting15.5should succeed.- Special version encodings like
255.255,99.0, or65535.255.255should be parsed as upper limits, not literal decimal values.