[BUG] deepspeed fails with torch 2.5 due to module._parameters is a dict, no longer a OrderedDict

[skydoorkai]

Describe the bug
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1904, in forward
   module._parameters._in_forward = False
AttributeError: 'dict' object has no attribute

To Reproduce
Steps to reproduce the behavior:
use torch 2.5 with ds.

torch 2.4 uses OrderedDict, which can add _in_forward attribute.
torch 2.5 uses dict for _parameters, and attribute adding is not supported.

Expected behavior
No failure.

ds_report output
Please run ds_report to give us details about your setup.

Screenshots
If applicable, add screenshots to help explain your problem.

System info (please complete the following information):

• OS: [e.g. Ubuntu 18.04]
• GPU count and types [e.g. two machines with x8 A100s each]
• Interconnects (if applicable) [e.g., two machines connected with 100 Gbps IB]
• Python version
• Any other relevant info about your setup

Launcher context
Are you launching your experiment with the deepspeed launcher, MPI, or something else?

Docker context
Are you using a specific docker image that you can share?

Additional context
Add any other context about the problem here.

[manojks1999]

@skydoorkai can I work on this?

[loadams]

Hi @skydoorkai - some of our CI run with torch 2.5 without failures, can you confirm the torch version or if you're on the latest nightly build?

Edit: I see our latest tests with torch-nightly are passing as well, so I'll be curious on your version or if you are hitting something that isn't covered in the unit tests.

@manojks1999, are you looking to contribute to DeepSpeed?

[loadams]

@skydoorkai - following up on this, could you provide the information from above?

@manojks1999 - are you able to repro this?