change ctpelok77's packages to build via dockerhub

[Rowan441]

resolves #100

Packages pulling with shub:

Name	URL	Dockerhub present?
cerberus.sif	shub://ctpelok77/fd-red-black-postipc2018	Yes
delfi.sif	shub://ctpelok77/delfi-deploy	Yes
forbiditerative.sif	shub://ctpelok77/fi-singularity	Yes
kstar.sif	shub://ctpelok77/kstar	Yes
enhsp-2018.sif	shub://enricos83/enhsp:2018	No
enhsp-2019.sif	shub://enricos83/enhsp:2019	No
enhsp-2020.sif	shub://enricos83/enhsp:2020	No

ctpelok77 packages are already on docker: https://hub.docker.com/u/ctpelok77

Once other packages are on dockerhub we can get them by replacing:

- singularity pull --name downward.sif shub://aibasel/downward
+ apptainer build {name}.sif docker://{username}/{name}:latest

Example dockerhub image: https://hub.docker.com/r/aibasel/downward

Example shub -> dockerhub conversion PR: #107

[haz]

@ctpelok77 Can you confirm that the dockerhub versions are runnable just the same as the shub?

@hstairs Can you throw some stuff up on dockerhub?

[ctpelok77]

Ok, I will need more details. What dockerhub images are we talking about, all four?
I never tried to build apptainer from a docker image, so cannot confirm or deny anything. Further, I am not sure anymore that the versions are the same. It's been 3 years...

[haz]

@ctpelok77 We just want to be sure that the docker versions are the same as the shub ones (i.e., we can run it in the same way). We've already taken care of the right syntax to build the singularity .sif from the docker image.

[ctpelok77]

@haz I can barely remember what I had for breakfast today...
Sorry, I have no idea whether these are the same versions as .sif files. I would expect them to be the same or very close, but cannot guarantee that they are the same.

[haz]

Heh, no worries. You had eggs.

@Rowan441 : can you post the output of trying to run those newly changed planners?

[Rowan441]

Output:

(planutils) root@0637efd5996e:~$ delfi domain.pddl pfile11.pddl plan.txt
FATAL:   "domain.pddl": executable file not found in $PATH

Dockerhub Images likely not working atm

[ctpelok77]

I guess I didn't have eggs, ha? :)

[haz]

Lol. Can you hook us up with anything you might have written, @ctpelok77 ? Dockerfiles, singularity scripts, etc. We can look over what's there and re-compile / upload things to the dockerhub.

[ctpelok77]

There have been some changes since then and some more will follow (e.g., for IPC 2023).
For now:

• The older K* (to be updated soon) https://github.com/ctpelok77/kstar
• Delfi recipes https://github.com/ctpelok77/delfi-deploy
• ForbidIterative https://github.com/IBM/forbiditerative. (just updated the old Docker, not 100% sure it is correct though, not tested)
• Cerberus is now integrated into ForbidIterative, look at the docker for copied scripts.

[haz]

Looks like all the elements are there.

@ctpelok77 Where would you like the official Dockerfiles? In those repos? If so, we can fork and put in a PR.

@Rowan441 Can you take the lead on modifying the Dockerfiles you can find in the linked repos? You can see an example entrypoint [here] that calls the planner on the docker image rather than just running bash.

Once you have a Dockerfile ready to go, you can test it like this...

# Build it
$ docker build -t <planner-name> .

# Repackage it as an apptainer .sif
$ apptainer build <planner-name>.sif docker-daemon://<planner-name>:latest

# Test the .sif
$ ./<planner-name>.sif domain.pddl problem.pddl

Just replace <planner-name> with whatever makes sense. You should be able to try all this inside the planutils docker image, if you don't want to install apptainer locally.

[ctpelok77]

I guess that depends on which one.

Delfi is a legacy code on bitbucket, which I cannot (and wouldn't) touch. I have no problem with keeping that github repo with deployment recipes, and feel free to PR, I will accept blindly.

K* - we are rewriting it and will publish a different version soon. ICAPS reviewers kinda interfered with our plans, but hopefully its significantly improved version will be publicly available soon. When it will be available, similarly to Delfi, it will be available in a different repo. So, you can PR into this repo and I will also accept blindly.

For the other two, we are maintaining that repo, but I don't think I use the docker recipe file. So, I will have to review your PRs carefully. There, I actually would suggest to move it into a separate folder. Now, I presume you don't use singularity recipes anymore. What is your solution for running different configurations (planners) on a single code?

[haz]

@ctpelok77 So do you want to create a docker recipe repo, and we'll just do it all there? If the repos are public, then the git clone can be in the recipe itself (no need to locate it inside the repo). Might be cleaner, since these docker files aren't for a development environment (which can lead to confusion, as we've found with planutils).

If there's multiple configurations on the same build, then you wrap with command-line parameters. Great example here:

• https://github.com/AI-Planning/planutils/blob/main/planutils/packages/lama-first/run
• https://github.com/AI-Planning/planutils/blob/main/planutils/packages/lama/run
• https://github.com/AI-Planning/planutils/blob/main/planutils/packages/downward/run

If it needs separate compile, then a separate docker is fine. Kind of how prp and poprp operate currently (latter is a branch of former, and each have their own recipe).

[ctpelok77]

Nah, I don't think a separate repo is needed. If you want to use a single docker recipe for all ForbidIterative planners as well as Cerberus, just make changes to the current recipe.

[haz]

Status on this, @Rowan441 ?

[haz]

@ctpelok77 What is the proper entry point for the generic forbiditerative? We'd like to just alias things like this, but it's a bit unclear from here what the right one would be. Looking for something like this FD entrypoint. It have something to do with plan.py?

[ctpelok77]

We just made a change to use it as a package: https://github.com/IBM/forbiditerative#using-as-a-package

You can see an example here https://github.com/IBM/forbiditerative/blob/main/plan-packaged-topk.py
I haven't changed the docker yet though...

[haz]

What's it install when you go the package route? I reckon too much expected locally, so still having it "dockered" would be wise.

There a build path that forgoes having CPLEX license locally? Don't think you'd be free to build + host a version with CPLEX embedded inside (unless the usage agreement has changed recently).

[ctpelok77]

Sure, let's go the dockered route. You can see examples here https://github.com/IBM/forbiditerative/blob/main/plan_topk.sh

Most of these do not require CPLEX. The only two that do are described here https://github.com/IBM/forbiditerative#dependencies
We can just forgo those.

In principle, the readme in that repo is somewhat informative.

[haz]

@Rowan441 Did your merged PRs address this?