Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dagster-k8s] infer image from dagster/image tag in PipesK8sClient #26629

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[dagster-k8s] infer image from dagster/image tag in PipesK8sClient #26629

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion python_modules/libraries/dagster-k8s/dagster_k8s/pipes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
extract_message_or_forward_to_stdout,
open_pipes_session,
)
from dagster._core.storage.tags import DOCKER_IMAGE_TAG
from dagster_pipes import (
DAGSTER_PIPES_CONTEXT_ENV_VAR,
DAGSTER_PIPES_MESSAGES_ENV_VAR,
Expand Down Expand Up @@ -379,7 +380,7 @@ def run(
context (Union[OpExecutionContext, AssetExecutionContext]):
The execution context.
image (Optional[str]):
The image to set the first container in the pod spec to use.
The image to set the first container in the pod spec to use. If the Dagster run has the `dagster/image` tag set, it will be used as the default when running one container in the pod.
command (Optional[Union[str, Sequence[str]]]):
The command to set the first container in the pod spec to use.
namespace (Optional[str]):
Expand Down Expand Up @@ -424,6 +425,10 @@ def run(
) as pipes_session:
namespace = namespace or _detect_current_namespace(self.kubeconfig_file) or "default"
pod_name = get_pod_name(context.run_id, context.op.name)

if not base_pod_spec or len(base_pod_spec.get("containers", [])) == 1:
image = image or context.dagster_run.tags.get(DOCKER_IMAGE_TAG)
Comment on lines +429 to +430
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could also fetch this using the same logic the run launcher does, but the run launcher does also set this tag, so I suppose either works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to assert in some fashion that it is non-null after doing this?


pod_body = build_pod_body(
pod_name=pod_name,
image=image,
Expand Down