Allow to process USD files to use relative paths when published

[BigRoy]

This PR has moved

This PR has been superseded by ynput/ayon-usd#70

---

Changelog Description

This adds a publish plug-in that is able to remap all paths inside the USD layer to a relative path. As such, by doing so - this means that all published USD files will be using solely relative paths to other published paths.

Additional info

This could allow you to 'archive' your project or share the publishes (while maintaining the project's folder structure for the publishes) and share those files with others, allow them to load it - without having to remap anything and without using an asset resolver.

This uses UsdUtils.ModifyAssetPaths which has existed for at least six years in USD, see commit. So should be safe to use nowadays across the board, but for completeness it was added in USD 19.03 so this feature requires USD 19.03+

Example result when loading an asset back in:

The top non-relative path is just the 'path' I loaded into Houdini to check the file contents; so that not being relative is entirely correct.

Testing notes:

1. Enable the plug-in in settings: ayon+settings://core/publish/USDOutputProcessorRemapToRelativePaths/enabled. (It is disabled by default.)

2. Publishes from Maya and Houdini (and others that generate USD publishes with asset paths to other files) should now remap all asset paths to relative paths.

Tested:

• Houdini 20.5.320 on Windows
• Houdini x.x.x on Linux
• Maya 2025.2 on Windows

[BigRoy]

One could argue that this should really be in ayon-usd. And I'd say, yes. Easy to move there. @dee-ynput @mkolar @antirotor thoughts?

However, that addon currently always downloads USD libs and configures USD resolvers and whatnot - which means there's so much bloat if you just want this. (However, IF the USD libs are available to tray publisher then technically this remapping would then also work for tray publisher which could be a strong combination.)

I'm just wondering - should we make that 'downloading' of USD libs and configuring of the asset resolver optional via settings in the ayon-usd repository? So that it's much wider ayon-usd tools, potentially running without the resolver and without configuring lakefs?

[BigRoy]

We may need to test this functionality with asset paths with <UDIM> token set and see if the UsdUtils.ModifyAssetPaths is affected by a bug/change in USD. See:

• UsdUtils.ExtractExternalReferences converts <udim> to 1001 in 23.11 PixarAnimationStudios/OpenUSD#3173
• https://forum.aousd.org/t/usdutils-extractexternalreferences-converts-udim-to-1001-in-23-11/1669

[Lypsolon]

One could argue that this should really be in ayon-usd. And I'd say, yes. Easy to move there. @dee-ynput @mkolar @antirotor thoughts?

However, that addon currently always downloads USD libs and configures USD resolvers and whatnot - which means there's so much bloat if you just want this. (However, IF the USD libs are available to tray publisher then technically this remapping would then also work for tray publisher which could be a strong combination.)

I'm just wondering - should we make that 'downloading' of USD libs and configuring of the asset resolver optional via settings in the ayon-usd repository? So that it's much wider ayon-usd tools, potentially running without the resolver and without configuring lakefs?

tbh i think we should move it there.
i actually think we should move most settings there, currently its hardly possible to remember where all the usd settings are.
i think it would be very nice if you can configure Usd in the Usd tab and not it like 4 tabs by now ?

[Lypsolon]

not sure if my testing was correct but the image there shows my output.

what is did:

• open houdi 20.5 create a sop lop and stored the data on my desktop
• then i created a usd also on the Desktop that references the other one
• then i referenced the second usd and created an Usd asset from it. (using the ayon Uris is enabled btw)
• but the output ends up with the correct remapping just not the Uris are remaped.

[BigRoy]

• open houdi 20.5 create a sop lop and stored the data on my desktop

• then i created a usd also on the Desktop that references the other one

• then i referenced the second usd and created an Usd asset from it. (using the ayon Uris is enabled btw)

• but the output ends up with the correct remapping just not the Uris are remaped.

So - what the code does for now is:

• It does not relatively remap AYON entity URIs. I assumed that if you were writing entity URIs to begin with you'd prefer that over any of the other filepaths.
• The tmp/dell.usd I suspect is referenced INSIDE of a file you loaded manually here. We're not writing or changing those files - we're only making relative links inside of our publishes, not on existing files on disk. As such - that indeed does not turn relative.

Also note that if you were to load USDs from a different drive it would also fail to make it relative of course. A warning would then be logged to the publish report for that particular case. This is the code used to 'detect' the drive.

With that explained, I actually feel your particular example is working exactly as it should. However, it'd be better to also test without AYON Entity URIs to confirm the asset contributions, etc. also remap accordingly. (Again, it'd only remap the new contributions because the older ones would still be using the entity URIs)

[Lypsolon]

• open houdi 20.5 create a sop lop and stored the data on my desktop
• then i created a usd also on the Desktop that references the other one
• then i referenced the second usd and created an Usd asset from it. (using the ayon Uris is enabled btw)
• but the output ends up with the correct remapping just not the Uris are remaped.

So - what the code does for now is:

* It does **not** relatively remap AYON entity URIs. I assumed that if you were writing entity URIs to begin with you'd prefer that over any of the other filepaths.

* The `tmp/dell.usd` I suspect is referenced INSIDE of a file you loaded manually here. We're not writing or changing those files - we're only making relative links inside of our publishes, not on existing files on disk. As such - that indeed does not turn relative.

Also note that if you were to load USDs from a different drive it would also fail to make it relative of course. A warning would then be logged to the publish report for that particular case. This is the code used to 'detect' the drive.

With that explained, I actually feel your particular example is working exactly as it should. However, it'd be better to also test without AYON Entity URIs to confirm the asset contributions, etc. also remap accordingly. (Again, it'd only remap the new contributions because the older ones would still be using the entity URIs)

okay then i can approve this. i assumed you would remap URIs because your PR says that you want to send the files to an external person, and that would then need the resolver no ?

[BigRoy]

okay then i can approve this. i assumed you would remap URIs because your PR says that you want to send the files to an external person, and that would then need the resolver no ?

Correct - but this is about shipping those projects that are enabled like this from the beginning. It does not "fix" existing files on disk. (It's not hard to fix them with a post-script though if needed, as this code nicely shows)

I can add a setting to also remap entity URIs with this processor - but once you're doing that you're better off disabling the generation of them in the first place.

[Lypsolon]

okay then i can approve this. i assumed you would remap URIs because your PR says that you want to send the files to an external person, and that would then need the resolver no ?

Correct - but this is about shipping those projects that are enabled like this from the beginning. It does not "fix" existing files on disk. (It's not hard to fix them with a post-script though if needed, as this code nicely shows)

okay so this is just for the current publish then?
i assumed i could do a workflow like.

1. do all my work with uri and special paths etc
2. when i need to send it to an external company i then publish a "relative" version that maybe gets packaged into a folder and can then send it ?

but yeah not having that makes sens i mean it would break the files if we make them with relative paths in the publish.

I can add a setting to also remap entity URIs with this processor - but once you're doing that you're better off disabling the generation of them in the first place.

nah if its not for sending the file to an external "company" then this would be a bad idea, just imagine accidentally having it on and all you :latest url get to be paths.

[Lypsolon]

dose its job.
tested on Linux Houdini 20.5

[BigRoy]

2. when i need to send it to an external company i then publish a "relative" version that maybe gets packaged into a folder and can then send it ?

That'd be just a call to USD Utils API to localize the assets if you really wanted to do just that. But it's indeed not what this PR is about. Thanks for checking.

[BigRoy]

Closing this PR in favor of PR on ayon-usd here: ynput/ayon-usd#70