Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add IntoPyObjectExt trait #4708

Merged
merged 8 commits into from
Nov 19, 2024
Merged

add IntoPyObjectExt trait #4708

merged 8 commits into from
Nov 19, 2024

Conversation

davidhewitt
Copy link
Member

Possibly related to #4703

I was wondering how we might be able to simplify uses of IntoPyObject in generic code. This PR proposes a helper IntoPyObjectExt trait which adds some methods which erase the concrete python type and force creation of PyErr results.

Overall this seems like a good amount of simplification at call sites, so I'll probably use this in pydantic-core if we don't land this here until 0.24.

Draft because the docs are unfinished and there's also a ton more locations internally this can be used. The kids are sick and need me to comfort them a lot tonight so I think I'm done for the day.

@@ -180,6 +180,9 @@ pub trait IntoPy<T>: Sized {
///
/// It functions similarly to std's [`TryInto`] trait, but requires a [GIL token](Python)
/// as an argument.
///
/// The [`into_pyobject`][IntoPyObject::into_pyobject] method is designed for maximum flexibility and efficiency; it
/// - allows for a concrete Python type to be returned (the [`Target`][IntoPyObject::Target] associated type)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to finish this list, had to run from my pc just now

/// This is typically only useful when the resulting output is going to be passed
/// to another function that only needs to borrow the output.
#[inline]
fn into_bound_object_py_any(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this method is open to bikeshedding, I think I'm happy with this but there's probably a better one.

Copy link
Contributor

@ngoldbaum ngoldbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we still want to tell users about BoundObject in the IntoPyObject docs in the meantime, but it looks like trait probably covers most uses I've seen in downstream code, so the docs I'm adding in #4703 will probably need an update in this PR

Ok(obj) => Ok(obj.into_any()),
Err(err) => Err(err.into()),
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think into_unbound_py_any would also probably be useful (there's a customer in Coroutine::new it looks like).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable to me; it certainly is common to .unbind() in pydantic-core (so that data can be stored in structs, mostly).

@davidhewitt
Copy link
Member Author

I realise that this is non-breaking to add in 0.23.1 as long as we delay prelude addition to 0.24. So while I think something like this is helpful both internally and for users, there is no need to rush a merge. (Especially as users can just copy this trait into their crate downstream in the meanwhile.)

@davidhewitt
Copy link
Member Author

@Icxolu would love to hear your thoughts on this idea. Do you see alternative ways we could offer simple conveniences on top of IntoPyObject?

@Icxolu
Copy link
Contributor

Icxolu commented Nov 16, 2024

I was thinking about this for a while now. I agree we should offer some convieniences here, this patter does come up alot.
The two ways I see are the extension trait proposed here or adding them directly as default methods in IntoPyObject. Adding them to IntoPyObject would allow overwriting them, but I'm not really sure if we want to allow that. Is there optimization potential if we would? It would also add additional complexity to the trait, which is another reason to go with the extension (at least for now) given that the adoption of IntoPyObject seems difficult for users.

I'm still struggeling a bit with the names. The current ones feel either a bit imprecise or a bit long to me (but I don't have suggestions that I'm completely happy with either). From a functionality point of view I would have 3 methods ins mind:

  • -> PyResult<Bound<PyAny>> maybe into_bound_any?
  • -> PyResult<Py<PyAny>> maybe into_py_any?
  • -> PyResult<Self::Output> no idea how to name this one

Other combinations feel less common and should probably go through into_pyobject directly.

@davidhewitt
Copy link
Member Author

Thanks, I am happy to go with those names with one small adjustment; I went for into_bound_py_any instead of into_bound_any because I think it's helpful for the method to have py somewhere in the name.

Agreed about keeping it a separate trait, I had the same thinking.

I named your option 3 into_pyobject_or_pyerr, which I think at least hints at its difference to the normal into_pyobject method in the right sort of way.

I think there's some more additional documentation and internal uses to add, if you're happy with this current set of names I'll seek to finish up and push later.

@davidhewitt davidhewitt marked this pull request as ready for review November 17, 2024 07:50
@davidhewitt
Copy link
Member Author

I also think there is probably space for a separate IntoPyObjectInfallible trait, but I would prefer to wait with that to see how the 2024 edition changes interactions with ! and Infallible.

Copy link
Contributor

@Icxolu Icxolu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I think the diff shows nicely already that this does decrease boilerplate and improves readability. I'm reasonable happy with the names and if other suggestions come up at some point, we can always refactor then.

I also think there is probably space for a separate IntoPyObjectInfallible trait, but I would prefer to wait with that to see how the 2024 edition changes interactions with ! and Infallible.

I agree. I think min_exhaustive_patterns (1.82+) already helps alot with this and we can afford to wait and see what the 2024 editions brings to the table.

Copy link
Contributor

@Icxolu Icxolu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks much nicer overall. Just a small doc suggestion.

guide/src/conversions/traits.md Outdated Show resolved Hide resolved
@davidhewitt davidhewitt added this pull request to the merge queue Nov 19, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 19, 2024
@Icxolu Icxolu added this pull request to the merge queue Nov 19, 2024
Merged via the queue into PyO3:main with commit 4be4759 Nov 19, 2024
45 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants