Measure Dart SDK size overhead

[denrase]

#skip-changelog

📜 Description

Measure Sentry Dart SDK size overhead in a simple dart binary.

💡 Motivation and Context

Relates to #2437

[denrase]

@vaind Would it make sense to measure binary size difference with getsentry/action-app-sdk-overhead-metrics for non iOS/Android files?

[vaind]

@vaind Would it make sense to measure binary size difference with getsentry/action-app-sdk-overhead-metrics for non iOS/Android files?

Is it possible to determine that if we extract the packaged app? For end-users it likely doesn't make much sense - it's actually the whole dart+native that is added to their app (i.e. what we show at the moment), but if the info is there and it helps developers then yeah, it would be possible to add it to the PR comment.

[github-actions]

iOS Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1250.91 ms	1269.90 ms	18.98 ms
Size	8.42 MiB	9.89 MiB	1.47 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
90db9ff	1277.18 ms	1283.69 ms	6.51 ms
1a93825	1257.25 ms	1261.55 ms	4.30 ms
905bf99	1240.84 ms	1271.47 ms	30.63 ms
2d74010	1264.45 ms	1268.42 ms	3.97 ms
e5b744f	1250.82 ms	1284.46 ms	33.64 ms
3e9fb0e	1262.49 ms	1280.65 ms	18.16 ms
7045efc	1246.45 ms	1262.24 ms	15.80 ms
09eab47	1246.88 ms	1267.71 ms	20.83 ms
8609bd8	1267.16 ms	1291.39 ms	24.22 ms
33d0587	1262.16 ms	1270.50 ms	8.34 ms

App size

Revision	Plain	With Sentry	Diff
90db9ff	8.10 MiB	9.08 MiB	1004.27 KiB
1a93825	8.28 MiB	9.34 MiB	1.05 MiB
905bf99	8.38 MiB	9.74 MiB	1.36 MiB
2d74010	8.32 MiB	9.38 MiB	1.05 MiB
e5b744f	8.09 MiB	9.07 MiB	1001.19 KiB
3e9fb0e	8.15 MiB	9.12 MiB	989.77 KiB
7045efc	8.38 MiB	9.78 MiB	1.40 MiB
09eab47	8.38 MiB	9.77 MiB	1.40 MiB
8609bd8	8.28 MiB	9.34 MiB	1.06 MiB
33d0587	8.29 MiB	9.38 MiB	1.09 MiB

Previous results on branch: ci/dart-only-metrics

Startup times

Revision	Plain	With Sentry	Diff
f497ca3	1257.37 ms	1286.13 ms	28.76 ms
b16ca72	1257.23 ms	1273.63 ms	16.40 ms

App size

Revision	Plain	With Sentry	Diff
f497ca3	8.38 MiB	9.78 MiB	1.40 MiB
b16ca72	8.42 MiB	9.83 MiB	1.40 MiB

[github-actions]

Android Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	443.00 ms	523.93 ms	80.93 ms
Size	6.46 MiB	7.48 MiB	1.02 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
dd76eef	461.37 ms	540.55 ms	79.18 ms
e8603bb	377.51 ms	444.77 ms	67.26 ms
0a82a1e	321.02 ms	393.82 ms	72.80 ms
72eeb80	421.38 ms	520.81 ms	99.44 ms
2f8f173	323.31 ms	373.29 ms	49.97 ms
d10745a	318.86 ms	374.82 ms	55.96 ms
84bc635	395.57 ms	464.23 ms	68.66 ms
8932ece	309.56 ms	377.28 ms	67.72 ms
633cf2e	289.36 ms	340.38 ms	51.02 ms
4b943a1	348.17 ms	437.15 ms	88.98 ms

App size

Revision	Plain	With Sentry	Diff
dd76eef	6.35 MiB	7.40 MiB	1.05 MiB
e8603bb	6.34 MiB	7.28 MiB	967.80 KiB
0a82a1e	6.15 MiB	7.11 MiB	981.82 KiB
72eeb80	6.35 MiB	7.42 MiB	1.06 MiB
2f8f173	5.94 MiB	6.95 MiB	1.01 MiB
d10745a	6.26 MiB	7.20 MiB	956.08 KiB
84bc635	6.34 MiB	7.28 MiB	968.41 KiB
8932ece	6.16 MiB	7.13 MiB	1000.89 KiB
633cf2e	5.94 MiB	6.92 MiB	1001.53 KiB
4b943a1	6.34 MiB	7.28 MiB	968.41 KiB

Previous results on branch: ci/dart-only-metrics

Startup times

Revision	Plain	With Sentry	Diff
b16ca72	467.70 ms	560.80 ms	93.10 ms
f497ca3	471.16 ms	519.36 ms	48.20 ms

App size

Revision	Plain	With Sentry	Diff
b16ca72	6.46 MiB	7.48 MiB	1.01 MiB
f497ca3	6.49 MiB	7.57 MiB	1.08 MiB

[vaind]

@vaind Would it make sense to measure binary size difference with getsentry/action-app-sdk-overhead-metrics for non iOS/Android files?

Is it possible to determine that if we extract the packaged app? For end-users it likely doesn't make much sense - it's actually the whole dart+native that is added to their app (i.e. what we show at the moment), but if the info is there and it helps developers then yeah, it would be possible to add it to the PR comment.

Ah, I misunderstood. I thought you meant to measure the sentry dart/flutter SDK overhead on top of the native sentry SDK (which maybe be stored in different libs in the app) but you actually meant adding non-android/ios stuff to the action. Yes, that would definitely make sense and shouldn't be too hard to do. I've had that in mind when building the action in the first place so it should work out of the box or with minor adjustments.

[denrase]

Ah, I misunderstood. I thought you meant to measure the sentry dart/flutter SDK overhead on top of the native sentry SDK (which maybe be stored in different libs in the app) but you actually meant adding non-android/ios stuff to the action. Yes, that would definitely make sense and shouldn't be too hard to do. I've had that in mind when building the action in the first place so it should work out of the box or with minor adjustments.

This pr is now building with a script and failing if we are over a simple threshold value. Would it make sense to extend the action in a separate PR? That way we won't be blocked here.

[buenaflor]

needs to be changed for linux