Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Parallel KFR execution of the same simple pipeline #1109

Open
2 tasks done
roytman opened this issue Mar 7, 2025 · 0 comments · May be fixed by #1110
Open
2 tasks done

[Bug] Parallel KFR execution of the same simple pipeline #1109

roytman opened this issue Mar 7, 2025 · 0 comments · May be fixed by #1110
Assignees
Labels
bug Something isn't working

Comments

@roytman
Copy link
Member

roytman commented Mar 7, 2025

Search before asking

  • I searched the issues and found no similar issues.

Component

Library/kfp

What happened + What you expected to happen

When super pipeline contains parallel execution of the same simple pipeline, KFP Runs dd not have different names.
Moreover, the Runs write to the same output directory, overwriting each other's results.

Example of parallel execution:

Image

Additionally, if parent_path_suffix is empty, the super pipeline output directory name starts with _

Reproduction script

Create the above pipeline.

Anything else

No response

OS

Ubuntu, Red Hat Enterprise Linux (RHEL)

Python

3.11.x

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
@roytman roytman added the bug Something isn't working label Mar 7, 2025
@roytman roytman self-assigned this Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant