Skip to content

Conversation

ryspark
Copy link
Contributor

@ryspark ryspark commented Aug 18, 2025

Main contribution is TaskConditionedHelios, an alternative to the Helios encoder that handles loading/forward pass of task embeddings. For now, these task embeddings are computed via a sentence embedding model on top of information extracted from each task's rslearn config. This depends on functionality from this helios PR, which may take some time to get merged, so merging this should be held off as well.

Second contribution is a MoE-style ViT moved over from rslearn, used for the decoder portion of a finetuned model.

Lastly, adds high-level documentation for using the multitask code across helios, rslearn, and rslearn_projects.

Other changes include extra tests, a custom checkpointer that allows restoring partial checkpoints, and updates to one_off_projects.

@ryspark ryspark marked this pull request as ready for review September 10, 2025 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant