Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report brms refitting models #374

Open
jgeller112 opened this issue Jun 8, 2023 · 7 comments
Open

Report brms refitting models #374

jgeller112 opened this issue Jun 8, 2023 · 7 comments
Labels
3 investigators ❔❓ Need to look further into this issue

Comments

@jgeller112
Copy link

Maybe I am missing something, but is there a way to avoid always refitting brms models using the report function? I have pretty large models and using report will either error out or run forever.

@strengejacke
Copy link
Member

@rempsyc @mattansb @DominiqueMakowski I'm not sure, but I guess this is due to computing Bayes factors when retrieving the performance-table? We should try to find the bottle neck here, maybe using the profile package.

@jgeller112 do you have a model you can share, so we can try to reproduce your issue? Else, maybe you find a model out of our pre-compiled models from here: https://easystats.github.io/circus/reference/index.html and just name one of those example models that behaves similarly.

@mattansb
Copy link
Member

I think this is because report standardizes coefficients using the refit method (which it 100% should not do since it does not adjust the priors accordingly).

@jgeller112
Copy link
Author

jgeller112 commented Jun 27, 2023

Here you go:

library(tidyverse)
library(report)
fit_c <- read_rds("https://osf.io/hmqpc/download")
report(fit_c)

@strengejacke
Copy link
Member

If I understand the profiling right, one issue is logLik():

image

And another bottle neck is rstan::summary():

image

Given that already the simple call to summary() takes ~ 7 seconds on my machine, this looks like the issue is due to the very complex model. The profile-file is attached (I used profvis), maybe we find ways to remove some of those time-consuming things.

report_brms.zip

@DominiqueMakowski
Copy link
Member

I think this is because report standardizes coefficients using the refit method (which it 100% should not do since it does not adjust the priors accordingly).

yes, the standardization method should be changeable to something else than "refit" (?)

@strengejacke
Copy link
Member

@jgeller112 Can you then try to run report(fit_c, include_effectsize = FALSE)?

@jgeller112
Copy link
Author

I get

Error in report_table.brmsfit(x, include_effectsize = FALSE, ...) : 
  formal argument "include_effectsize" matched by multiple actual arguments

@rempsyc rempsyc added the 3 investigators ❔❓ Need to look further into this issue label Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 investigators ❔❓ Need to look further into this issue
Projects
None yet
Development

No branches or pull requests

5 participants