Is there a way to invoke pipelines/command one by one when multiple changes happened in multiple path?

[JoeyDon]

steps:
  - label: "Detecting which mono-repo has changed"
    plugins:
      - chronotc/monorepo-diff#v1.3.0:
          diff: "git diff --name-only HEAD~1"
          watch:
            - path: "folder-a/"
              config:
                label: ":pipeline: Continue blue-alice pipeline"
                command: "buildkite-agent pipeline upload ./folder-a/.buildkite/pipeline.yml"
            - path: "folder-b/"
              config:
                label: ":pipeline: Continue dwh-store pipeline"
                command: "buildkite-agent pipeline upload ./folder-b/.buildkite/pipeline.yml"

Hey guys, my root directory's pipeline.yml looks like that. I have sub-directories having it's own defined pipeline.ymland all invoked from this root directory's pipeline.yml.

Is there a way that I can ensure each pipeline run in order? Currently I tried concurrency group and - wait in each sub-directoriy's pipeline.yml but no luck.

Would really appreciate your thinking and replies!

[ronaldmiranda]

@JoeyDon, Looking at the code, there is a environment variable called BUILDKITE_PLUGIN_MONOREPO_DIFF_WAIT that enables a function called add_wait to put - wait on every step builded by the plugin, probably suits in your case.

[JoeyDon]

@ronaldmiranda thanks for looking into for me.

local wait
wait=${BUILDKITE_PLUGIN_MONOREPO_DIFF_WAIT:-true}

I grabbed these snippet from add_wait. But it looks like the logic is if no variable BUILDKITE_PLUGIN_MONOREPO_DIFF_WAIT found, it will just give wait a true value, isn't it?

[ronaldmiranda]

@JoeyDon Yeah, you're right. it should be added the wait by default.

[LColman-SEEK]

Hey @JoeyDon , not sure if you've looked into it (or if it's exactly appropriate,) but BK itself has the depends_on and key fields that you could use within your folder-a and folder-b pipelines. You could define a key in folder-a's pipeline.yml and use this value in the folder-b pipeline.yml as a depends_on.
I know it's a bit clunky, and makes the folder-level pipelines not reusable/composable in other pipelines, etc. But should be able to give you a guaranteed order of execution. (Don't get me wrong, I would much rather have this bug fixed in the plugin too, but as a stop gap you can use depends_on).

[orolega]

Any update on this?

[adikari]

@orolega can you try the built-in depends_on as suggested by @LColman-SEEK and see if that works for you? If not then I will explore this issue further.

[orolega]

The built-in depends_on would not be a viable solution unfortunately. The ideal outcome would be like a wait step between each defined path block