fetch: does not work with ssh even though scp/rsync/sftp do. #43
Labels
Comments
|
Hi @Jalmenara , thank you for detailed report! Would you be willing to try using https://github.com/ronf/asyncssh directly to see if you could narrow it down to either asyncssh or dvc up the stack? |
|
Hello @efiop . Thank you for the quick answer. Indeed, I think that I have found something with
Then, I adapted some basic examples of the docs and tested them:
Exception has occurred: SystemExit
SFTP operation failed: [Errno 2] No such file or directory: b''
OK, small update. I tested This led me to try to remove the end slash at the However, this did not solve the problem. How does exactly |
|
Additional info: |
|
@Jalmenara Thanks for trying it out! We use asyncssh through https://github.com/fsspec/sshfs and to get a file we use this https://github.com/fsspec/sshfs/blob/a62fd30cfcf55ef74345a0cc398f5779a1577ffa/sshfs/spec.py#L171 But your original traceback seems to point to just |
|
@efiop , thanks for your time! |
|
@Jalmenara I mean your initial log seems to point us to connection failing: Are you sure you are still getting the same kind of error? |
|
Ah, alright. Sorry for my bad understanding... @efiop Yes, i am still getting the same error, even after the tests with |
|
If you run dvc with The i.e. assuming the PDB session shows you the traceback for you can just do to see the value of host (and same for the other variables at that point in the traceback) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
Description
dvc fetch&pullcommands do not work with ssh remote even though the same path works withscp,rsyncandsftp+getReproduce
I do not see a clear way to make this problem reproducible. In fact, I tried to post this as a Discord question, but my description was too long for a message. Hopefully it can be understood properly. The situation is as follows:
My team and I are facing a problem when using dvc ssh/sftp tools.
We are setting up a dvc remote shared by two companies to work on a project: clientA & contractorB (I belong to the latter).
The remote is hosted by the clientA at a location accessible by us through ssh.
There is an intermediate proxy server, but we have sorted that out using the
ProxyJumpoption in~/.ssh/config, like this:The dvc remote is stored at
destination.We have also configured the ssh keys on the servers of clientA, so that no password prompts are needed.
On the
dvcside, we have configured the remote in our repo with the following.dvc/config:However, the
dvc fetch/pullcommands fail, with the following prompt:The first thing I did was ensuring that the paths were written correctly.
For instance, I removed the
:in the url of the.dvc/config, between "destination" and "/work":This did not solve the problem.
Next, to discard issues with the ssh/sftp connections, I decided to copy manually
the
/work/projects/models-bin/folder from clientA to my contractorB'scomputer using three different methods:
scp,rsyncandsftp.The three methods work properly: the "dvc-like" files appear at contractorB side (e.g.,
models-bin/b0/26324c6904b2a9cb4b88d6d61c81d1).This is what led me to think that the issue
might be on the dvc side, and not so much on the connection or the paths. Note that
the path used in the three manual methods is exactly the same (copy-pasted).
Expected
Be able to locate the files.
Environment information
Output of
dvc doctor:Additional Information:
Output of
dvc fetch -v:The text was updated successfully, but these errors were encountered: