Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[explore] nested config with defaults #26603

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

jamiedemaria
Copy link
Contributor

@jamiedemaria jamiedemaria commented Dec 19, 2024

Summary & Motivation

from
https://dagsterlabs.slack.com/archives/C06QU0WBCQ5/p1734624150011189

It looks like if you set nested config with defaults, those dont get respected.

Example

 class NestedConfig(dg.Config):
     a: str
     b: int

 class AssetConfigSetDefaultsForNested(dg.Config):
     nested: NestedConfig = Field(default=NestedConfig(a="a", b=1))
     c: int = 1

@dg.asset
def example_with_asset_config_defaults(context, config: AssetConfigSetDefaultsForNested) -> None:
     context.log.info(f"nested.a: {config.nested.a}, nested.b: {config.nested.b}, c: {config.c}")

When you try to materialize the asset, you have to fill in values for nested in the launchpad

Making this PR with a test case demonstrating the behavior as a place to start documenting the behavior and investigating solutions since I likely won't be able to get a fix in before going on PTO.

One thought I had was that this might be considered an anti-pattern, but pydantic supports this

from pydantic import BaseModel


class Nested(BaseModel):
    a: str = "a"
    b: int


class Outer(BaseModel):
    foo: Nested = Nested(b=1)

print(Outer())
# foo=Nested(a='a', b=1)

How I Tested These Changes

Changelog

Insert changelog entry or delete this section.

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

field_type = pydantic_field.annotation
field_type = (
pydantic_field.annotation
) # here by just passing the field_type is where we lose any default values
if safe_is_subclass(field_type, Config):
inferred_field = infer_schema_from_config_class(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was that we'd pass the default values to infer_schema_from_config_class and then propogate those down the recursion until they reach the intended non-Config fields. We'd have to deal with overrides, but that seems simple enough - the default set at the higher level would override one set on a lower model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant