🐝 collect session replays via Sentry

[bnjmacdonald]

Collect session replays via Sentry for 1% of cookie-consenting user sessions. Closes owid/analytics#155.

Implementation notes and privacy considerations.

Remaining todos before merging:

• Update privacy policy to mention Sentry usage
• apply for Sentry nonprofit discount
• temporarily add SENTRY_DSN to staging env (then remove after merging – we don't need to waste precious replay budget on staging sites going forward). owid/ops#233
• If Sentry nonprofit discount not approved yet, start Sentry free trial (so that we aren't capped at 50 session replays)
• add SENTRY_DSN to prod env (ops:templates/owid-admin-prod/admin-env.secret)

[owidbot]

Quick links (staging server):

Site	Admin	Wizard

Login: ssh owid@staging-site-add-session-replays

• Site-screenshots: https://github.com/owid/site-screenshots/compare/add-session-replays
• SVG tester: https://github.com/owid/owid-grapher-svgs/compare/add-session-replays

SVG tester:

Number of differences (default views): 0 ✅
Number of differences (all views): 0 ✅

Edited: 2024-11-01 01:40:56 UTC
Execution time: 1.19 seconds

[rakyi]

Sorry, if you already discussed this before, but just a quick fly by review:

• We should be probably using @sentry/react instead of @sentry/browser to get better contextual data
• If we keep using Sentry we should stop using Bugsnag, but this can wait until we confirm it works as expected, let's just not forget about it

[rakyi]

Capturing replays on error seems even more useful than non-errors. Is there a good reason to disable it?

[bnjmacdonald]

It's more so the opposite for me, as I don't want our budget of session replays to get inadvertently consumed by a few pages that raise a lot of the same error over and over (e.g. we end up with 1k monthly session replays on /search out of our 5k budget just b/c of errors).

Obviously, keeping replaysOnErrorSampleRate low would help avoid this problem, but it was easier for now to just set this to 0 to get a clear baseline of how well the 5k monthly replays are distributed across our pages (without having to worry about how much errors skew the distribution of replays).

[bnjmacdonald]

• We should be probably using @sentry/react instead of @sentry/browser to get better contextual data

thanks Martin – done ✅

[larsyencken]

Looks good, no obvious issues with the integration from my end. We can obviously tweak the sampling rates later as needed.