Given nginx configuration slows down uploads over https

[namandureja]

Describe the bug
The given nginx configuration in the official docs - nginx.conf slows down uploads over https. I setup tusd on my virtual machine and after proxying it through nginx, I noticed significant drop in upload speeds (a factor of 20). Initially I thought that it was happening due to some issue in ssl encryption or proxy buffering by nginx. But, after a bit of experimenting, it turned out that this line was the root cause -
listen 443 http2 ssl;
The 'http2' configuration slowed down the uploads by a huge factor. Once I removed it, the speeds were back to normal again, as fast as on http.
Not sure if anyone else has faced this issue, but I thought it would be helpful for anyone running into this.

To Reproduce
Steps to reproduce the behavior:

1. Run tusd behind a proxy (nginx).
2. Enable ssl and listen to 443 port with http2 configuration enabled.
3. Test the upload speed.
4. Disable the http2 configuration.
5. Test the upload speed again.

Setup details
Please provide following details, if applicable to your situation:

• Operating System: Linux, amd64
• Used tusd version: v2.4.0
• Used tusd data storage: AWS S3
• Used tusd configuration: tusd -port -s3-bucket= -behind-proxy -disable-cors;
• Used tus client library: tested with both uppy and tus-js-client

[Acconut]

Thanks for the report! I remember hearing something similar some time ago, but there were no concrete results at the time. We haven't noticed a slowdown from using HTTP/2, but we are also using HAProxy in our deployments, not Nginx. When looking for information on this, one can find many similar reports, for example:

• https://forum.seafile.com/t/http2-drastically-slows-down-up-and-download/15472
• https://serverfault.com/questions/979694/enabling-http-2-makes-the-site-much-slower-in-nginx
• https://stackoverflow.com/questions/33658302/why-http-2-is-slower-than-plain-https
• Slow uploads with HTTP/2 nextcloud/documentation#9578

It seems as if the default configuration for HTTP/2 on Nginx is not optimized for uploads. So this is not a bug in tusd, but rather the result of a missing configuration, as some people report tuning the configuration helped improve upload speed.

The last link from above also links to a CloudFlare article, which dives into the topic of upload speed with HTTP/2. I haven't read it fully yet, but it seems as if it provides some potential hints to optimize upload speed. I want to look more into this, but maybe you can also read into it a bit.

[Chaoses-Ib]

We encountered the same problem. The upload speed is only about 20% when HTTP/2 is enabled with Nginx v1.27.

The CloudFlare article says they sent the patch to NGINX upstream. But I couldn't find either the commit or the code in the repo of Nginx. Maybe it's declined for some reason.

[Acconut]

Overall, this seems be a problem with the proxy and not tusd directly. We use HAProxy in production and have added this config to improve HTTP/2 performance:

    # for slow h2 uploads. See https://github.com/haproxy/haproxy/issues/293 for more context.
    tune.h2.initial-window-size 2097120

I'm not sure if Nginx's http2 module provides a similar setting: http://nginx.org/en/docs/http/ngx_http_v2_module.html

I cannot tell if the patch from CloudFlare was accepted or integrated in another way into Nginx. Maybe it's worth to ask Nginx about this problem directly. FYI, I also contacted the author to see if they have more insights: https://x.com/tus_io/status/1859544062962913582