[pythonic resources] Partial config support for resources

[benpankow]

Summary

(Marking this as ready to review, but this should probably bake a bit longer, I'm a bit hesitant to merge a late PR like this).

Introduces the ability to partially configure a ConfigurableResource. The remaining config must then be specified at runtime.

For example:

class MyResource(ConfigurableResource):
  foo_str: str
  bar_str: str

defs = Definitions(
    resources={"my_resource": MyResource.partial(foo_str="foo")
)

In this case, bar_str must be specified at runtime.

Test Plan

Introduced new unit tests.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Ignored Deployments

Name	Status	Preview	Comments	Updated
dagit-storybook	⬜️ Ignored (Inspect)			Mar 10, 2023 at 0:34AM (UTC)
dagster	⬜️ Ignored (Inspect)			Mar 10, 2023 at 0:34AM (UTC)

[benpankow]

• [pythonic resources] never treat partial resources as resolved #27210
• [pythonic config][alternate] properly propagate default values set at the top level for nested config classes #27209
• [pythonic resources] Partial config support for resources #12870 👈 (View in Graphite)
• [pythonic resources] Improve nested resource evaluation logic #14807
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

Deploy preview for dagster ready!

Preview available at https://dagster-lpqab6n0y-elementl.vercel.app

Direct link to changed pages:

[github-actions]

Deploy preview for dagit-core-storybook ready!

✅ Preview
https://dagit-core-storybook-qkrf5dchv-elementl.vercel.app

Built with commit 87305ba.
This pull request is being automatically deployed with vercel-action

[schrockn]

From brief perusal I agree with your assessment in the PR summary. We should wait until after the branch cut to merge this and release in a subsequent dot release. Let's focus on core docs and content until then.

[benpankow]

Reviving this PR based on user request - I think this functionality is useful enough to warrant trying to get it in. Thankfully most of the groundwork was already in our existing PartialResource class, so the changes required are pretty brief (though they do touch some core Config/Resources code).

[schrockn]

Overall makes sense. Let's try the suggestions on for size and then spin again and then merge.

[NellyWhads]

Hello folks,

Has there been any movement on this? I'm facing some issues with configure_at_launch() and the threads have led me here.