-
|
With regular Bayesian inference and tools like emcee I can assess the convergence of the process using a few methods like plotting and examining chain mixing, Gelman-Rubin statistic, and similar tools. Is there an equivalent method with pyABC? Or some number I should pay attention to (other than the distance)? |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments
-
|
Assessing posterior approximations at different generations (should become sharper over the generations), as well as effective population sizes (the number you are looking for) should be the first thing to look at. You find some visualisation examples here at the end of the example. More visualisation routines, e.g contour plots to compare prior and posterior distribution, you find here. You should also use your parameter samples to make simulations and compare it to your data for visual inspection and to compare summery statistics of your simulations and your data. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @arrjon. I usually try to have an ESS in the hundreds, but I guess even 50 would be ok if I need to estimate means and STDDEVs. I'm aware of the visualization methods but I'm not sure which one really gives me information about the convergence of the process. Comparing the data with simulations is reasonable but highly subjective. I think the answer is "there really is no convergence method for ABC algorithms"? |
Beta Was this translation helpful? Give feedback.
-
|
You can check the prior / posterio ratio as well (maybe that's what @arrjon pointed to above already). One can also do multiple start seeds in pyABC and see if parameters are estimated in narrow bounds (more heuristically). Not sure if there are any direct correlates to the Gelman-Rubin and Geweke diagnostic used for MCMC. |
Beta Was this translation helpful? Give feedback.
-
|
How could I check the prior/posterior ratio? I guess checking the evolution of the median+stddev of each parameter would also help to assess convergence. Is a plot like this included in the visualizations? |
Beta Was this translation helpful? Give feedback.
-
It's this one pyabc.visualization.plot_credible_intervals() |
Beta Was this translation helpful? Give feedback.
Assessing posterior approximations at different generations (should become sharper over the generations), as well as effective population sizes (the number you are looking for) should be the first thing to look at. You find some visualisation examples here at the end of the example.
More visualisation routines, e.g contour plots to compare prior and posterior distribution, you find here.
You should also use your parameter samples to make simulations and compare it to your data for visual inspection and to compare summery statistics of your simulations and your data.