Thanks for the feedback, @franklinselva. That repo structure pre-dates most people currently on the project, but @quarkytale might know why demos and simulation were split initially?

We're actually in the middle of a significant refactor aimed to resolve a similar issue between the space-ros and docker repos: #140
This will have an impact on what is currently the docker repo: #107
Which will then get renamed to stacks in keeping with ROS naming conventions: #160
How would this affect your proposal?

Since we're closing in on the quarterly release I've added a note to the agenda for the next technical committee meeting to make sure this gets reviewed: #179

@Bckempa Considering the issues and discussions mentioned, my proposal would be unaffected for most part. This proposal can be worked comfortably on considering a solution found for building moveit2 and nav2-related layers for the docker image. I see the osrf/space-ros:latest already in dockerhub.

Will I be able to join the next technical committee meeting by any chance?

Note that part of the previously linked issue is moving the demo docker files to the demo repo as well so it seems this is already some overlap and what you are proposing is a viable next step to improving the organization. Do I understand correctly?

Meeting attendance is managed by @EzraBrooks

@Bckempa I think there is no dependency either way. However, moving the docker files to the demo repository first will impose additional work before improving the organization (if docker files are expected to be merged to main). Reorganizing the higher level items first will help less changes to the docker files since the moveit2 and nav2 will be already staged and ready for demos package. I believe this is mostly done in the discussion #140.

My thought is if the demos are not released as packages through snap/docker/apt yet, we should get started with this and let the docker design take shape in the meantime. This way, we can work on both the parts in parallel.

I think making this change in a single PR would make sense. I can get started with this after the discussion in the meeting.

@Bckempa Adding more context from the discussion on docker. The image tags moveit and nav2 can be directly used for both the demo applications. This leaves the demo packages to be restructured independently.

why demos and simulation were split initially?

Mostly because earlier there were plans to add the same models to play with in different demo worlds. Since the goal of demos is changing, we could reorganize.

I think each demo should have it's own Dockerfile or docker-compose.yaml file so each can be built independently. Moving the Dockerfile from the docker repo to here and splitting it into two separate files would be a logical first step to doing that which wouldn't require much effort.

Agreed. Something like this for the docker entry points is expected

# Canardarm demo
FROM osrf/space-ros:moveit2
...

and

# Rover demo
FROM osrf/space-ros:nav2
...

I don't see if these two demos can be shared in the same environment. But using Makefile or docker-compose would help manage the docker images easily. I would opt for Makefile personally since we can set up a custom stream of sub-commands to help with demos.

I would opt for Makefile personally since we can set up a custom stream of sub-commands to help with demos.

Can you share some examples of what you'd like to use Makefiles for? I'm always hesitant to add a runtime dependency without a strong use-case.

Can you share some examples of what you'd like to use Makefiles for? I'm always hesitant to add a runtime dependency without a strong use-case.

I could think of a few usecases at the moment.

I want to mention that the proposal is structuring the demos repository as a mono repo. With that said, we should have multiple examples and demos for each robot (e.g. rover can have demos related to perception, navigation, exploration, mission planning, etc.) and to keep things modular, it is better to structure each section of demos in a separate ROS package.

Now, to build the demos, If I am concentrating only on perception for Rover, I am not required to build the mission planning and nav2-related examples of Rover or any other robots in general. This will save a lot of build time when the repository grows.

Something like this would be useful at this point,

make build curiosity_rover nav2_example # Nav2 related demo
make build curiosity_rover all # All demos of the rover

Even though it is not meant to be, I sometimes take the make command further for running the commands without tools like just.

Can you share some examples of what you'd like to use Makefiles for? I'm always hesitant to add a runtime dependency without a strong use-case.

I could think of a few usecases at the moment.

I want to mention that the proposal is structuring the demos repository as a mono repo. With that said, we should have multiple examples and demos for each robot (e.g. rover can have demos related to perception, navigation, exploration, mission planning, etc.) and to keep things modular, it is better to structure each section of demos in a separate ROS package.

Now, to build the demos, If I am concentrating only on perception for Rover, I am not required to build the mission planning and nav2-related examples of Rover or any other robots in general. This will save a lot of build time when the repository grows.

Something like this would be useful at this point,

make build curiosity_rover nav2_example # Nav2 related demo
make build curiosity_rover all # All demos of the rover

Even though it is not meant to be, I sometimes take the make command further for running the commands without tools like just.

why dont we use Earthfiles for something like this? they mix makefile functionality with docker functionality in a good way.

To address this, I believe the question to be analyzed is how the demos should be built and run from this point of change? Should we still have all the demos prebuilt inside image?

My idea here is to build a base docker image for a robot or a family of robots and use makefiles to build demos dynamically within the containers instead of building directly into images.

why dont we use Earthfiles for something like this? they mix makefile functionality with docker functionality in a good way.

Just to ensure everyone has the context of how we got here; like @asimonov mentioned on #107 (comment), not everyone was sold on Earthly so we decided to use it initially only for the base image build and keep the demos on Docker. Recently there has been more acceptance for moving the demos over as well which would automate many of the higher level build logic @franklinselva is referring to, but there was fairly strong consensus that at least one demo showing a vanilla docker setup should be maintained.

To address this, I believe the question to be analyzed is how the demos should be built and run from this point of change?

I agree this is the right question to ask.

My idea here is to build a base docker image for a robot or a family of robots and use makefiles to build demos dynamically within the containers instead of building directly into images.

I personally haven't given much thought to the demo workflow, but the system I've seen others like @mkhansenbot work toward was a preference for running multiple images over building a monolithic image. There are some major advantages to that at the cost of complexity.

I think we have considerable acceptance now. Should I start working on this change?

Should I start working on this change?

You don't need permission to work on things; however, I'm still not clear on the exact scope and timing of this change.

We're still progressing on our big build-system and repo organization migration, tracked by #107. Currently we are finishing #177 including addressing the consequences like space-ros/docker#160, and then propagating the change through to the other repositories space-ros/docker#162.

Are you proposing completing the reorg proposed above which would address space-ros/demos#27 and space-ros/demos#28 or something else? If you are what does that top-level docker file look like and did we resolve the monolithic vs disparate docker file discussion?

You are more than welcome to open a PR with your vision for comment if you feel that is easier, or if you'd prefer for the issues listed above to filter though so you can assess their impact we can try to hammer out the design here a bit more.

Having worked with the build system closely, I think most of the issues listed in #107 don't need to be carried down. #177 is already handled by #186 to an extent which needs to be split down to stick with the guidelines and NASA standards (I will still need some help on this).

To expand on the situation for #107, I have created a new issue #188 which should help determine the changes needed before proceeding.

This is what I would propose to be done in order

• Lightweight SpaceROS Images and Variants #186 & Adjust main spaceros image to the new build approach #177
• Discuss Plans for forking Nav2 and Moveit2 #188
• Decide the changes for nav2 and moveit2
• Fix Earthly/Dockerfile dichotomy #107
• Integrate docker into the demo workspace

This leaves the other changes of the demo to be performed in parallel.

I believe the above list is essential and has ended up as a part of restructuring the organization's content.

@franklinselva super minor nit, but would you mind updating the issue title to be a bit more specific? Every time I triage I forget exactly what it is referring to since the current wording is very broad and since it is hosted in the space-ros repo seems to imply work to be done here rather than docker/demos/simulation, etc.

Sure. I will do that. We can also move this issue if it is easier.

I think it is fine to keep it in this repo since it touches so many, just the title update helps when sorting through all the open issues. Thanks!