Using torch compile

[AndrewMead10]

Since we are using accelerate, Is it possible to use torch compile by setting dynamo_backend="INDUCTOR"? I am trying to get it working and it looks like the model gets unallocated and then the code hangs. Only change i made was

accelerator = Accelerator(
        gradient_accumulation_steps=training_args.gradient_accumulation_steps,
        log_with=training_args.report_to,
        project_config=accelerator_project_config,
        kwargs_handlers=[ddp_kwargs],
        dynamo_backend="INDUCTOR"
    )