Toward Cost-efficient Adaptive Clinical Trials in Knee Osteoarthritis with Reinforcement Learning

Paper: arXiv

Directory Structure

The project expects the following directory structure under the root_path specified in config.yaml:

/Data/OAI/                    # root_path
├── MetaData/                                   # metadata_path
│   └── OAICompleteData_ASCII/
├── Dataframe/                                  # dataframe_path
└── Images/                                     # images_path
    └── ResizedImages/

The paths are configured in config.yaml

KL Prediction File Requirements

The project requires KL grade predictions generated by a trained model. These predictions (either probabilities or logits) must be placed in the dataframe_path directory. This file should be named KL_predictions.pkl (pickle format).

Training the agent

To train the DQN agent with custom cost parameters, use the following command:

python dqn/run_agent.py cost.cost_convert_r=2 cost.hospital_cost=0.5 cost.true_dm_coef=0.3 cost.late_fl_coef=0.5

Command Parameters

The command uses Hydra configuration override syntax to modify parameters of the reward function:

• cost.cost_convert_r: $c$, the monetary loss of 1mm of articular cartilage (MLAC), calculated from TKR cost
• cost.hospital_cost: $\lambda$, data acquisition cost
• cost.true_dm_coef: $\beta/c$, the positive reward for the correct dismiss
• cost.late_fl_coef: $\alpha$, discounted parameter for late visit Please refer to equations in paper for further explainations.

You can override any configuration parameter from config.yaml using the parameter=value syntax.

Testing the agent

To evaluate the trained agent, use the following command:

python dqn/eval_agent.py model_test_path=<model checkpoint> cost.cost_convert_r=2 cost.hospital_cost=0.5 cost.true_dm_coef=0.3 cost.late_fl_coef=0.5