Enable performant rsync filetransfer

[consideRatio]

We currently have jupyterhub-ssh the Python package + Helm chart, and we have the jupyterhub-sftp part of that Helm chart that could in practice be its own Helm chart.

@yuvipanda describes that even if we solve #41, using rsync like that won't be very performant. Perhaps could the jupyterhub-sftp service be made more general, such as jupyterhub-filetransfer and be a service directly exposing some file transfer services like rsync, scp, and sftp? Maybe... Let this issue represent the wish for having a performant rsync.

[yuvipanda]

For jupyterhub-sftp, we want to allow file transfers but not arbitrary code execution. Unlike jupyterhub-ssh, jupyterhub-sftp does not interact with a running user's notebook server, but instead does file operations in its own container with the home directories mounted. So arbitrary code execution here is basically a RCE, and we don't want that.

Right now, we use the internal-sftp subsystem to provide sftp, and this works ok.

However, we want to provide support for rsync (#55), scp and other file transfer mechanisms too. There's no built-in support for these protocols in sshd.

scponly is (was?) a popular program designed exactly for this use case. Instead of acting as a subsystem, it forces acting as a shell - consuming any parameters sent in by the user, and allowing only scp and rsync. Assuming there are not any RCEs in rsync or scp, this should work ok? We still will use a chroot for additional protection. See scponly's SECURITY file for other precautions we need to take. Particularly, we'll need to provision a chroot inside our docker image that has just the bits needed for rsync / scp to function, and nothing else.

scponly is an unmaintained pile of C, and I'd like to not use it. We need to either write ourselves a small script that can do this shell filtering (only allow rsync, scp commands), or find a maintained version of scponly.

[fxp0]

Hello!
Do you guys maybe know any working solution/workaround to make rsync work? Trying to make an integration between PyCharm and jupyter pod and this issue is looks like a blocker 😞

[yuvipanda]

Unfortunately nope and I won’t have time to work on this for a few months at least. Sorry