Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implemement .devpodignore #1337

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+41 −2
Open

Implemement .devpodignore #1337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7f96b50
Implemement .devpodignore
bkneis Oct 28, 2024
c8c6967
Don't return error for rpc
bkneis Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions pkg/agent/tunnelserver/tunnelserver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"strings"

"github.com/loft-sh/devpod/pkg/agent/tunnel"
Expand All @@ -21,6 +22,7 @@ import (
provider2 "github.com/loft-sh/devpod/pkg/provider"
"github.com/loft-sh/devpod/pkg/stdio"
"github.com/loft-sh/log"
"github.com/moby/buildkit/frontend/dockerfile/dockerignore"
perrors "github.com/pkg/errors"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
Expand Down Expand Up @@ -370,8 +372,18 @@ func (t *tunnelServer) StreamWorkspace(message *tunnel.Empty, stream tunnel.Tunn
return fmt.Errorf("workspace is nil")
}

// Get .devpodignore files to exclude
excludes := []string{}
f, err := os.Open(filepath.Join(t.workspace.Source.LocalFolder, ".devpodignore"))
if err == nil {
excludes, err = dockerignore.ReadAll(f)
if err != nil {
t.log.Warnf("Error reading .devpodignore file: %v", err)
}
}

buf := bufio.NewWriterSize(NewStreamWriter(stream, t.log), 10*1024)
err := extract.WriteTar(buf, t.workspace.Source.LocalFolder, false)
err = extract.WriteTarExclude(buf, t.workspace.Source.LocalFolder, false, excludes)
if err != nil {
return err
}
Expand All @@ -392,8 +404,18 @@ func (t *tunnelServer) StreamMount(message *tunnel.StreamMountRequest, stream tu
return fmt.Errorf("mount %s is not allowed to download", message.Mount)
}

// Get .devpodignore files to exclude
excludes := []string{}
f, err := os.Open(filepath.Join(t.workspace.Source.LocalFolder, ".devpodignore"))
if err == nil {
excludes, err = dockerignore.ReadAll(f)
if err != nil {
t.log.Warnf("Error reading .devpodignore file: %v", err)
}
}

buf := bufio.NewWriterSize(NewStreamWriter(stream, t.log), 10*1024)
err := extract.WriteTar(buf, mount.Source, false)
err = extract.WriteTarExclude(buf, mount.Source, false, excludes)
if err != nil {
return err
}
Expand Down
17 changes: 17 additions & 0 deletions pkg/agent/workspace.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
provider2 "github.com/loft-sh/devpod/pkg/provider"
"github.com/loft-sh/log"
"github.com/mitchellh/go-homedir"
"github.com/moby/buildkit/frontend/dockerfile/dockerignore"
)

var extraSearchLocations = []string{"/home/devpod/.devpod/agent", "/opt/devpod/agent", "/var/lib/devpod/agent", "/var/devpod/agent"}
Expand Down Expand Up @@ -303,6 +304,22 @@ func CloneRepositoryForWorkspace(

log.Done("Successfully cloned repository")

// Get .devpodignore files to exclude
f, err := os.Open(filepath.Join(workspaceDir, ".devpodignore"))
if err != nil {
return nil
}
excludes, err := dockerignore.ReadAll(f)
if err != nil {
log.Warn(".devpodignore file is invalid : ", err)
return nil
}
// Remove files from workspace content folder
for _, exclude := range excludes {
os.RemoveAll(filepath.Join(workspaceDir, exclude))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is expected. We're using a bind mount for the devcontainer with local docker and this would delete the excludes on the users machine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but only from the content folder, i.e. ~/.devpod/context/..., which I think is fine? since other devpod functionality could use this so deleting would help "ignore" the files elsewhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but only from the content folder, i.e. ~/.devpod/context/...

Yep, we still need to make an exception for local docker and local folder because there it's a direct bind mount to the origin folder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes your right, my PR description was wrong I had not tested local dir and local docker. Only local dir with kubernetes, which uses the stream RPCs. The deletion here only happens with a git source (since it's in CloneRepositoryForWorkspace). I'm not sure how we can support .devpodignore for the local docker and local dir use case. Unless I symlinked or moved the original folder elsewhere, there will be no way around the bind mount mounting ignored files. But maybe this is fine? Since if you are using local docker and dir, you proboably want everything mounted? What do you think? Can you see a way of supporting?

}
log.Debug("Ignore files from .devpodignore ", excludes)

return nil
}

Expand Down
Loading