Etherpad grains not working after upgrade to 0.304

[garrison]

Starting today, I've been unable to get grains running Etherpad to load properly. I am using Etherpad app version 1.8.14-2. (However, I will note that grains that have been shared with me using an older, pre-FundOSS Etherpad version continue to work.) Other apps seem to be working still (at least the ones I've tried so far, including EtherCalc).

Specifically, when I open an Etherpad grain (or when anyone I've shared the grain with opens it), I see an error, such as

An error occurred while loading the pad
SyntaxError:literal not terminated before end of script in
https://ui026adf21285f40fbd605f4ada00334b6.garrison.sandcats.io/javascripts/lib/ep_etherpad-lite/static/js/ace2_common.js?callback=require.define&v=e34d6eef at line 6387

There might be slight variations on this. For instance, I might reload and see

An error occurred while loading the pad
ReferenceError:jQuery is not defined in
https://ui27245dbb54bd9b17f5ac63442b0ebc8e.garrison.sandcats.io/static/plugins/ep_comments_page/static/js/jquery.tmpl.min.js at line 1

This happens with both existing grains and newly created grains.

When I search the internet for this error, this is the only thing that comes up: https://wiki.documentfoundation.org/Website/Etherpad#Weird_errors_when_connecting.3F. However, it seems to happen regardless of browser and regardless of browser extensions installed.

I have no idea whether Sandstorm itself is the problem, but it is the only thing I can think of. Sandstorm has more or less always "just worked" for me, so it is very odd to see something break. I find it interesting that the timing seems to match when my server upgraded to 0.304, for which the release notes say

Cap'n Proto has had several internal optimizations. We don't expect a significant impact on Sandstorm but let us know if there's any fishy behavior changes.

I'm not certain it's related, but this behavior is certainly fishy.

I haven't tried reproducing on a development Sandstorm server but plan to try when I get a chance.

[garrison]

I tried and was unable to reproduce this on a development server. In fact, I was able both to create new Etherpad grains and to restore my existing grains from zip backups just fine on that server.

[zenhack]

I can confirm. Looking at the changelog:

v0.304 (2022-12-10)

    Cap'n Proto has had several internal optimizations. We don’t expect a significant impact on Sandstorm but let us know if there’s any fishy behavior changes.

I'm going to see if I can reproduce this on a local dev build and if so whether rolling back the capnp version fixes it.

[zenhack]

Yeah, rolling back to the latest stable capnp release fixes it; see #3677.

[kentonv]

I've pushed a new release rolling back capnp, do sudo sandstorm update or wait 24 hours and it should be fixed, let me know if not.

[zenhack]

Seems to work for me.

[ocdtrekkie]

I think we should probably keep this issue open to track the problem blocking us from updating capnp for the moment.

[garrison]

Works now for me too. Thank you!