Generalize command resampling

[lorenwel]

Description

This introduces the concept of a ResamplingManager and ResamplingTerms. This generalizes command resampling beyond the currently implemented fixed time resampling.

Every CommandTerm now owns a ResamplingManager which it calls during the compute() call to determine whether commands should be resampled.
A ResamplingManger can contain multiple RewardTerms to obtain diverse behavior.

All current example RL environment configs which used a command other than the NullCommand where changed to use the new FixedFrequency resampling term, which replicates the current behavior. The NullCommand now simply receives now ResamplingTerm and is therefore never resampled, as desired.

However, this is a breaking change for people who created their own RL configs outside of the Orbit main branch. Luckily, this is very easy to fix with a single line code change:

resampling_time_range=(10.0, 10.0)--> resampling={"fixed_frequency": FixedFrequencyCfg(resampling_time_range=(10.0, 10.0))},

I also added a new RandomChance resampling term as an example, which resamples environments with a fixed probability.

Fixes #223

Type of change

• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update (probably)

Checklist

• I have run the pre-commit checks with ./orbit.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have run all the tests with ./orbit.sh --test and they pass
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[Mayankm96]

Did a very high-level parse of the code. Overall, this is an interesting feature. However, it seems that resampling time term inherited from a manager term class is an overkill.

Wouldn't it be easier to have it as a function instead? From what I see, the following function signature should suffice.

def my_samping_func(env_ids: Sequence[int], *args) -> torch.Tensor

[lorenwel]

A simple function would suffice for some applications but even the current fixed time resampling mechanism needs persistent storage you cannot have inside a function. It needs to keep track of the time left.
If we go function-based, all these additional variables would need to be put into the environment or command class. This would lead to everyone putting lots and lots of stuff in there for the case that someone might use it for resampling, instead of having it nicely compartmentalized inside of a dedicated class.

So I definitely think a resampling class is necessary.
Using a manager is nice because we can use multiple resampling terms to achieve compound behavior. We can for example resample randomly every step but at the latest after some seconds:

resampling={
    "fixed_frequency": FixedFrequencyCfg(resampling_time_range=(10.0, 10.0)),
    "random_choice": RandomChoiceCfg(resampling_probability=(0.01)),
}

But if you think it would be better to create a dedicated resampling class for this behavior, I could be convinced that the manager is not necessary :)

[lorenwel]

@Mayankm96 Any movement on this? Happy to resolve all conflicts, if you are still interested in this.