Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation: clarification of the estimate argument for each metric #489

Open
RandallJEllis opened this issue Oct 29, 2024 · 1 comment

Comments

@RandallJEllis
Copy link

Problem
I spent a day trying to understand the cumulative_dynamic_auc function because I was consistently getting values in the 0.15-0.3 range. After experimenting, I realized that for the estimate argument, I was using the output of predict_survival_function (in a RandomSurvivalForest model) instead of predict, and switching this fixed the issue and gave me reasonable AUCs.

Proposed solution
The documentation for cumulative_dynamic_auc is not very descriptive on what kind of outputted predictions (e.g., probabilities, estimated times-to-event) the estimate argument should be, and I think this could be clearer for all metrics.

@sebp
Copy link
Owner

sebp commented Nov 2, 2024

Thanks for reporting this issue. The current description for estimate is:

Estimated risk of experiencing an event of test data. If estimate is a 1-d array, the same risk score across all time points is used. If estimate is a 2-d array, the risk scores in the j-th column are used to evaluate the j-th time point.

It does mention that estimate should be a risk score.

Can you please elaborate which parts are not clear to you? Is it that the description isn't referencing predict?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants