chore(ci): Dynamically select available iPhone simulator for tests #5577

antonis · 2026-01-26T10:29:37Z

📢 Type of change

Bugfix
New feature
Enhancement
Refactoring

📜 Description

Instead of hardcoding device name and OS version, dynamically find the first
available iPhone simulator using xcrun simctl. This prevents failures when
specific device/OS combinations aren't available on CI runners.

💡 Motivation and Context

Fixes #5572

💚 How did you test it?

CI, I've run the native ios test twice and was 🟢

📝 Checklist

I added tests to verify changes
No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
I updated the docs if needed.
I updated the wizard if needed.
All tests passing
No breaking changes

🔮 Next steps

#skip-changelog

The OS=latest parameter in xcodebuild destination is unreliable on GitHub's macos-15 runners due to frequent simulator runtime deprecations. By omitting the OS version, xcodebuild automatically selects any available iPhone 16 simulator, making the tests more stable across runner image updates. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

github-actions · 2026-01-26T10:29:59Z

Semver Impact of This PR

⚪ None (no version bump detected)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).

This PR will not appear in the changelog.

_{🤖 This preview updates automatically when you update the PR.}

github-actions · 2026-01-26T10:46:00Z

Android (legacy) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	507.15 ms	528.52 ms	21.38 ms
Size	43.75 MiB	48.40 MiB	4.64 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`136effd`+dirty	398.57 ms	494.91 ms	96.34 ms
`d916aa3`	425.37 ms	427.02 ms	1.65 ms
`5691430`+dirty	433.98 ms	478.49 ms	44.51 ms
`4604da9`+dirty	366.44 ms	398.10 ms	31.66 ms
`b80b14f`+dirty	505.06 ms	534.32 ms	29.26 ms
`8e653ac`+dirty	360.28 ms	372.04 ms	11.76 ms
`a2bb688`+dirty	409.65 ms	410.45 ms	0.80 ms
`7480abe`+dirty	411.60 ms	405.81 ms	-5.78 ms
`9a81842`+dirty	412.23 ms	416.56 ms	4.33 ms
`ec14be7`+dirty	403.50 ms	411.46 ms	7.96 ms

App size

Revision	Plain	With Sentry	Diff
`136effd`+dirty	43.75 MiB	47.99 MiB	4.23 MiB
`d916aa3`	17.75 MiB	20.15 MiB	2.41 MiB
`5691430`+dirty	43.75 MiB	48.04 MiB	4.29 MiB
`4604da9`+dirty	17.75 MiB	19.74 MiB	2.00 MiB
`b80b14f`+dirty	43.75 MiB	48.04 MiB	4.29 MiB
`8e653ac`+dirty	17.75 MiB	19.75 MiB	2.00 MiB
`a2bb688`+dirty	17.75 MiB	19.70 MiB	1.95 MiB
`7480abe`+dirty	17.75 MiB	19.68 MiB	1.94 MiB
`9a81842`+dirty	43.75 MiB	48.08 MiB	4.33 MiB
`ec14be7`+dirty	17.75 MiB	19.69 MiB	1.94 MiB

Previous results on branch: antonis/native-ios-flakiness

Startup times

Revision	Plain	With Sentry	Diff
`efee771`+dirty	451.78 ms	488.42 ms	36.64 ms

App size

Revision	Plain	With Sentry	Diff
`efee771`+dirty	43.75 MiB	48.40 MiB	4.64 MiB

github-actions · 2026-01-26T10:46:10Z

Android (new) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	393.47 ms	422.44 ms	28.97 ms
Size	43.94 MiB	49.22 MiB	5.29 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`136effd`+dirty	451.30 ms	450.87 ms	-0.43 ms
`d751a5d`+dirty	341.61 ms	403.06 ms	61.45 ms
`5691430`+dirty	376.70 ms	401.58 ms	24.88 ms
`1226664`+dirty	377.65 ms	453.94 ms	76.29 ms
`b80b14f`+dirty	320.33 ms	376.06 ms	55.73 ms
`785ffb1`+dirty	380.65 ms	451.83 ms	71.18 ms
`8e653ac`+dirty	304.49 ms	308.84 ms	4.35 ms
`21c9e75`+dirty	356.73 ms	381.06 ms	24.33 ms
`161947d`+dirty	349.27 ms	435.09 ms	85.82 ms
`d1fd647`+dirty	374.46 ms	409.51 ms	35.05 ms

App size

Revision	Plain	With Sentry	Diff
`136effd`+dirty	43.94 MiB	48.81 MiB	4.88 MiB
`d751a5d`+dirty	7.15 MiB	8.41 MiB	1.26 MiB
`5691430`+dirty	43.94 MiB	48.87 MiB	4.93 MiB
`1226664`+dirty	7.15 MiB	8.46 MiB	1.30 MiB
`b80b14f`+dirty	43.94 MiB	48.87 MiB	4.93 MiB
`785ffb1`+dirty	7.15 MiB	8.42 MiB	1.27 MiB
`8e653ac`+dirty	7.15 MiB	8.46 MiB	1.31 MiB
`21c9e75`+dirty	7.15 MiB	8.42 MiB	1.27 MiB
`161947d`+dirty	7.15 MiB	8.43 MiB	1.28 MiB
`d1fd647`+dirty	7.15 MiB	8.43 MiB	1.28 MiB

Previous results on branch: antonis/native-ios-flakiness

Startup times

Revision	Plain	With Sentry	Diff
`efee771`+dirty	406.36 ms	429.67 ms	23.31 ms

App size

Revision	Plain	With Sentry	Diff
`efee771`+dirty	43.94 MiB	49.22 MiB	5.29 MiB

Instead of hardcoding device name and OS version, dynamically find the first available iPhone simulator using xcrun simctl. This prevents failures when specific device/OS combinations aren't available on CI runners. Also adds a debug step to list available simulators for troubleshooting. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

github-actions · 2026-01-26T11:19:32Z

iOS (new) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1202.75 ms	1207.89 ms	5.14 ms
Size	3.38 MiB	4.60 MiB	1.22 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`df1f7df`+dirty	1217.00 ms	1213.27 ms	-3.73 ms
`c771b48`+dirty	1189.46 ms	1191.15 ms	1.69 ms
`459a438`+dirty	1218.39 ms	1226.14 ms	7.75 ms
`55b77fc`+dirty	1213.08 ms	1218.19 ms	5.11 ms
`60cd796`+dirty	1206.08 ms	1213.36 ms	7.28 ms
`69602ce`+dirty	1230.59 ms	1230.84 ms	0.24 ms
`fdbea8b`+dirty	1196.19 ms	1212.09 ms	15.90 ms
`294387d`+dirty	1199.23 ms	1204.16 ms	4.93 ms
`3099014`+dirty	1234.04 ms	1243.22 ms	9.18 ms
`fa7bb7e`+dirty	1226.27 ms	1241.88 ms	15.61 ms

App size

Revision	Plain	With Sentry	Diff
`df1f7df`+dirty	3.44 MiB	4.59 MiB	1.15 MiB
`c771b48`+dirty	3.41 MiB	4.58 MiB	1.17 MiB
`459a438`+dirty	3.19 MiB	4.55 MiB	1.36 MiB
`55b77fc`+dirty	3.41 MiB	4.58 MiB	1.17 MiB
`60cd796`+dirty	3.44 MiB	4.67 MiB	1.23 MiB
`69602ce`+dirty	3.19 MiB	4.48 MiB	1.29 MiB
`fdbea8b`+dirty	3.41 MiB	4.59 MiB	1.18 MiB
`294387d`+dirty	3.41 MiB	4.59 MiB	1.18 MiB
`3099014`+dirty	3.19 MiB	4.55 MiB	1.36 MiB
`fa7bb7e`+dirty	3.19 MiB	4.58 MiB	1.39 MiB

Previous results on branch: antonis/native-ios-flakiness

Startup times

Revision	Plain	With Sentry	Diff
`efee771`+dirty	1214.66 ms	1214.33 ms	-0.33 ms

App size

Revision	Plain	With Sentry	Diff
`efee771`+dirty	3.38 MiB	4.60 MiB	1.22 MiB

github-actions · 2026-01-26T11:22:09Z

iOS (legacy) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1205.96 ms	1213.00 ms	7.04 ms
Size	3.38 MiB	4.60 MiB	1.22 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`df1f7df`+dirty	1219.21 ms	1224.28 ms	5.07 ms
`c771b48`+dirty	1224.02 ms	1219.72 ms	-4.30 ms
`459a438`+dirty	1222.12 ms	1214.60 ms	-7.53 ms
`55b77fc`+dirty	1223.71 ms	1224.12 ms	0.41 ms
`60cd796`+dirty	1204.87 ms	1202.71 ms	-2.17 ms
`69602ce`+dirty	1235.65 ms	1230.82 ms	-4.83 ms
`fdbea8b`+dirty	1210.38 ms	1211.96 ms	1.57 ms
`294387d`+dirty	1197.73 ms	1208.35 ms	10.61 ms
`3099014`+dirty	1226.75 ms	1236.56 ms	9.81 ms
`fa7bb7e`+dirty	1216.23 ms	1225.04 ms	8.81 ms

App size

Revision	Plain	With Sentry	Diff
`df1f7df`+dirty	3.44 MiB	4.59 MiB	1.15 MiB
`c771b48`+dirty	3.41 MiB	4.58 MiB	1.17 MiB
`459a438`+dirty	2.63 MiB	3.98 MiB	1.35 MiB
`55b77fc`+dirty	3.41 MiB	4.58 MiB	1.17 MiB
`60cd796`+dirty	3.44 MiB	4.67 MiB	1.23 MiB
`69602ce`+dirty	2.63 MiB	3.91 MiB	1.28 MiB
`fdbea8b`+dirty	3.41 MiB	4.59 MiB	1.18 MiB
`294387d`+dirty	3.41 MiB	4.59 MiB	1.18 MiB
`3099014`+dirty	2.63 MiB	3.98 MiB	1.35 MiB
`fa7bb7e`+dirty	2.63 MiB	4.01 MiB	1.38 MiB

Previous results on branch: antonis/native-ios-flakiness

Startup times

Revision	Plain	With Sentry	Diff
`efee771`+dirty	1186.53 ms	1187.67 ms	1.14 ms

App size

Revision	Plain	With Sentry	Diff
`efee771`+dirty	3.38 MiB	4.60 MiB	1.22 MiB

alwx · 2026-01-26T13:32:25Z

.github/workflows/native-tests.yml

-          DESTINATION: 'platform=iOS Simulator,OS=latest,name=iPhone 16'
        run: |
+          # Find first available iPhone simulator
+          DEVICE_ID=$(xcrun simctl list devices available iPhone -j | jq -r '.devices | to_entries[].value[] | select(.isAvailable == true) | .udid' | head -1)


One quick question: I noticed that this command does return "iPhone 16" with iOS 18.6 when I run it so it seems like there is no sorting applied there, at least from what I see when running this locally. Could it be that the destination will become some older simulator with some older version of iOS or is it practically impossible?

Great catch @alwx 👍 Updated with a253400
Let's see if everything works on CI now 🤞

Rerun the test a couple of times with the latest changes and stayed 🟢
@alwx Ready for another pass 🙇

Improves device selection to explicitly choose from the latest available iOS runtime, preventing tests from running on older iOS versions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

lucas-zimerman

It works locally, LGTM!

antonis added the ready-to-merge Triggers the full CI test suite label Jan 26, 2026

antonis changed the title ~~chore(ci): Fix flaky iOS native tests by removing OS=latest~~ chore(ci): Dynamically select available iPhone simulator for tests Jan 26, 2026

antonis marked this pull request as ready for review January 26, 2026 12:23

antonis requested review from alwx and lucas-zimerman as code owners January 26, 2026 12:23

antonis mentioned this pull request Jan 26, 2026

Support for applying scope attributes to logs #5579

Open

10 tasks

alwx requested changes Jan 26, 2026

View reviewed changes

chore(ci): Select iPhone simulator from latest iOS runtime

a253400

Improves device selection to explicitly choose from the latest available iOS runtime, preventing tests from running on older iOS versions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

antonis requested a review from alwx January 26, 2026 13:37

lucas-zimerman approved these changes Jan 26, 2026

View reviewed changes

antonis enabled auto-merge (squash) January 26, 2026 15:12

Uh oh!

chore(ci): Dynamically select available iPhone simulator for tests #5577

Are you sure you want to change the base?

chore(ci): Dynamically select available iPhone simulator for tests #5577

Uh oh!

Conversation

antonis commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📢 Type of change

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

🔮 Next steps

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Semver Impact of This PR

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Android (legacy) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: antonis/native-ios-flakiness

Startup times

App size

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Android (new) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: antonis/native-ios-flakiness

Startup times

App size

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

iOS (new) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: antonis/native-ios-flakiness

Startup times

App size

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

iOS (legacy) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: antonis/native-ios-flakiness

Startup times

App size

Uh oh!

alwx Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

antonis Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

antonis Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

lucas-zimerman left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

antonis commented Jan 26, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading