Root folders for projects

[sunix]

Is your feature request related to a problem? Please describe.
At the moment, the devfile declares projects that are usually git projects. These git projects are actually git repository. In these git repos, it happens that projects to be loaded in an IDE are actually located in subfolders of the git repo.

For instance: https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app should have node-frontend and quarkus-backend loaded as Project (Eclipse IDE terminology) or Workspace root (VSCode/Theia/Che terminology).

These projects or roots should be concidered and defined in the devfile.

Describe the solution you'd like
From eclipse-che/che#15347 (adapted to devfile 2)

Each projects defined in a workspace, would possibly have a list of folder that would be workspace roots of the Che workspace. /projects is the default one.

projects:

  - name: che-quarkus-demo
    git:
      location: 'https://github.com/sunix/che-quarkus-demo'
      branch: microservices
    roots: ['quarkus-backend', 'node-frontend']

  - name: nodejs-ex
    git:
      location: 'https://github.com/sclorg/nodejs-ex'

  - name: nodejs-ex-sunix
    git:
      location: 'https://github.com/sunix/nodejs-ex'
    roots: []

would add the 3 folders /projects/nodejs-ex, /projects/che-quarkus-demo/quarkus-backend, /projects/che-quarkus-demo/node-frontend as workspace root.

Of course if the user is using the command Add folder to workspace, the devfile of the workspace would need to be updated.

/projects/nodejs-ex-sunix is not added as a workspace root because of it has roots array empty.
/projects/nodejs-ex is added as a workspace root because it is the default behaviour if no roots element is defined.

  - name: nodejs-ex
    git:
      location: 'https://github.com/sclorg/nodejs-ex'

would be similar to

  - name: nodejs-ex
    git:
      location: 'https://github.com/sclorg/nodejs-ex'
    roots: ['.']

Describe alternatives you've considered
Using sparseCheckoutDirs, but user may want to keep the whole. sparseCheckoutDirs is useful for examples/samples repositories.

Additional context
Add any other context or screenshots about the feature request here.

[benoitf]

Hi Sun, what is the difference with eclipse-che/che#15347 ?

you don't want to use generic attributes in projects field ?

[sunix]

I would prefer to have that part of the devfile schema

[l0rd]

@sunix what's the use case for the empty project root:

  - name: nodejs-ex-sunix
    git:
      location: 'https://github.com/sunix/nodejs-ex'
    roots: []

The project gets cloned in /projects/ but the IDE ignores it?

[sunix]

@l0rd yes. It should not happen very often, but it is to clarify the behavior if it happens.

[l0rd]

I would not supporting this use case (maybe with an attribute?).

And when roots is not specified I would expect be equivalent to roots=[] and therefore to correspond to the top folder to be the workspace root.

Another comment is that having both roots and sparseCheckoutDir is confusing from a user point of view. I would add to this proposal the deprecation of sparseChekoutDir.

[sunix]

I would not supporting this use case (maybe with an attribute?).

And when roots is not specified I would expect be equivalent to roots=[] and therefore to correspond to the top folder to be the workspace root.

OK

Another comment is that having both roots and sparseCheckoutDir is confusing from a user point of view. I would add to this proposal the deprecation of sparseChekoutDir.

I don't think we should deprecate sparseCheckoutDir: these are doing different things and could be used together or not.
For git projects, sparseCheckoutDir is git related and maybe used for different reasons. roots would be related to the structure of your project.

• roots could be used for git projects with several components: for instance a git project with node-fronend folder and another quarkus-backend folder.
• sparseCheckoutDir are usually used in sample repos, where we want just to clone/retrieve a subfolder. For instance https://github.com/quarkusio/quarkus-quickstarts, we would want to clone only 1 folder.

[benoitf]

I think that for quarkus-quickstarts we mainly want to see one project in the IDE but having other quickstarts there could be helpful as well as you may want to open them
I also think it was a workaround waiting to have proper multi-root workspaces support.

[elsony]

Based on cabal discussion, we'll:

1. Remove sparseCheckoutDir from spec since we don't find a good use case for it.
2. roots will be a Che specific attribute since it is mostly for the UI

[sunix]

I am surprise that odo wouldn't use these kind of meta informations about projects/roots. For git repos that contain several projects in subfolders, this is a generic way to tell where the projects are.
In project like https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app, how would odo or che or eclipse or intellij know that the frontend is to be built from node-frontend and the backend from quarkus-backend?

[yangcao77]

@kadel Does Odo care about the project subfolders? The entire git repo will be cloned including the subfolders.

[kadel]

In project like https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app, how would odo or che or eclipse or intellij know that the frontend is to be built from node-frontend and the backend from quarkus-backend?

Why complicate it with roots?
I can simply have something like this

projects:
  - name: quarkus-reactjs
    git:
      location: 'https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app'


commands:
  - id: build-frontend
    exec:
      component: nodejs
      commandLine: "npm install"
      workingDir: ${PROJECTS_ROOT}/node-frontend
  - id: build-backend
    exec:
      component: java
      commandLine: "mvn package"
      workingDir: ${PROJECTS_ROOT}/quarkus-backend

Then I can go further and even create a composite command to build both.

  - id: build-all
    composite:
      commands:
        - build-frontend
        - build-backend
      group:
        kind: run
        isDefault: true

[kadel]

@kadel Does Odo care about the project subfolders? The entire git repo will be cloned including the subfolders.

odo currently doesn't use projects field at all. Odo always assumes that you are always working with the source code that you have locally. And it might not even be in a git repo.

[sunix]

OK fair enough :)
In classical IDEs, projects or roots are used to reference source code of a module. IDEs can't just rely on the commands to figure out where is the java project or the nodejs project and which one to activate. In a git repo, we may even not be interested in all the projects/folders for a particular development task.
These are also being used for watching filesystem changes on the selected folders ... not all the git repo.

[kadel]

I think that we should discourage people from using multi-project devfiles.
In the above example, there should ideally be two devfiles, one for the frontend and the other for backed.

From Che's point of view, one devfile should represent one workspace. Multi-workspaces shoudl be represented as opening multiple devfiles.
After all, CRD based on Devfiles is called DevWorkspace, it can get a little bit confusing if we are saying that DevWorkspace can define multiple Che workspaces.

[sunix]

I think we should not encourage nor discourage people to use one pattern or another.
Developers have their constraints. There is always a good reason to choose to have multiple project in a git repo and have one devfile related to that. It most of the time depends on the project life cycle, not on the project architecture. The term workspace is not the same depending on which community we are talking to. In Eclipse IDE, a workspace is a set of projects that could be span into several git repos.
We should provide something that works for their projects and not encouraging to them to change their projects so it fits our tools.
Our users would like to replicate what they have in production in a kind of devmode, having one workspace that let them code, build and run the frontend + the backend + the database is very common use case.

[kadel]

Agreed that this needs to be flexible to work with the existing projects no matter what is their structure. But at the same time we need to guide new users and establish best practices.
If we don't set best practices and recommended structure, new users will get lost.

Our users would like to replicate what they have in production in a kind of devmode, having one workspace that let them code, build and run the frontend + the backend + the database is very common use case.

This is true and possible only for small projects. Most of the developers won't work on the backend and frontend at the same time. Most of the time they won't be even maintained by the same development teams. Establishing the best practice where each service has its own devfile is important. This is how most of the developers that we target operate.
Of course, there will be exceptions, and Devfile will still work for them. But in this case, it can be a little bit more complicated to define things in Devfile spec. Devfile doesn't have to have and should not have special fields for every possible case.