[Feature] ConditionalPolicySwitch transform

[vmoens]

Stack from ghstack (oldest at bottom):

• [DRAFT] ppo chess with llm and ConditionalPolicySwitch to sunfish bot #2763
• -> [Feature] ConditionalPolicySwitch transform #2711
• [Example] Self-play chess PPO example #2709

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2711

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• ROCM Infra failures during checkout of PyTorch

❌ 10 New Failures, 6 Unrelated Failures

As of commit 11f7d8a with merge base dbc8e2e ():

NEW FAILURES - The following jobs have failed:

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_rl__3.9_cpu_
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_6 (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_rl__3.9_cu126_
• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t ad7bb14ce89a05edc2d94f6a5afa0efc2b9bb0eda3cedfa95fdf6aedf96b33a4 /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.11, 12.4) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_str2str_env_tokenizer_catframes
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t fe47214cdc3222abf715ad603b0201b78e1b1a6f0b7866efcdaa409d8aecdbdc /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  Process completed with exit code 1.
• Wheels / build-wheel-windows (3.11, 3.11) (gh)
• Wheels / build-wheel-windows (3.12, 3.12) (gh)
  This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v3. Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to download artifact(s): Artifact not found for name: pytorch_rl__3.9_cu124_
• Habitat Tests on Linux / tests (3.9, 12.4) / linux-job (gh) (trunk failure)
  AttributeError: _ARRAY_API not found
• Unit-tests on Linux / tests-optdeps (3.11, 12.4) / linux-job (gh) (trunk failure)
  test/test_transforms.py::TestTokenizer::test_transform_inverse
• Wheels / build-wheel-windows (3.10, 3.10.3) (gh) (trunk failure)
  This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v3. Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/
• Wheels / build-wheel-windows (3.9, 3.9) (gh) (trunk failure)
  This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v3. Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

This comment was automatically generated by Dr. CI and updates every 15 minutes.