Skip to content

import: remote file can be downloaded via dvc get but can't be downloaded with import #19

Open
@hv10

Description

@hv10

Bug Report

Description

When using a dvc repo as a data registry which uses an sshfs as remote we ran into issues importing data into depended repositories, specifically importing a directory which includes several files, while the directory itself is the object tracked added to the remote as a stage output.

Reproduce

  1. have a repo that uses sshfs as default remote (using ssh-key auth)
  2. init new repo
  3. dvc import a directory --> leads to PermissionError
  4. dvc get the same directory --> works as expected

Expected

dvc import and dvc get should both be able to pull the directory and files in question.

Environment information

  • there exists a ssh config containing the config with two keyfiles, one for the repo host (gitlab) and one for the ssh remote (our lab cluster)

Output of dvc doctor:

DVC version: 2.29.0 (pip)
---------------------------------
Platform: Python 3.10.6 on macOS-12.6-arm64-arm-64bit
Subprojects:
	dvc_data = 0.14.0
	dvc_objects = 0.5.0
	dvc_render = 0.0.11
	dvc_task = 0.1.3
	dvclive = 0.11.0
	scmrepo = 0.1.1
Supports:
	http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
	ssh (sshfs = 2022.6.0)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s3s1
Caches: local
Remotes: ssh, local
Workspace directory: apfs on /dev/disk3s3s1
Repo: dvc, git

Additional Information (if any):

Additional Info will be added when I tried the Debugging method from the Wiki.
Sadly I can not publish the output for dvc import -v <directory> for now as I would have to clean it from references to our cluster.
I know its fairly vague, but I do not have a clue where the issue could come from, therefore I would be happy for any pointers on where to look.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions