Problems uncovered by provisional StanHeaders

[bgoodri]

We are struggling to get a new version of StanHeaders onto CRAN and walker is one of several packages that we are having issues with.

First, walker has Makevars and Makevars.win files in src/ . Ordinarily, a package that uses rstantools like walker does should not have those files, since they get regenerated automatically by rstantools. That is why there is a comment at the top of them saying that they are autogenerated and should not be edited by hand. When those files are modified by hand, then rstantools does not overwrite them. However, that now means that the necessary changes to the build process are not picked up, which makes walker segfault when its namespace is loaded with the provisional StanHeaders. Just by deleting the src/Makevars and src/Makevars.win files, I was able to get walker to build and load, so I am not understanding why it was necessary to change them.

Second, after doing that, walker has some errors in its vignette and examples in situations where the provisional StanHeaders catches mistakes that are somehow not being caught with the StanHeaders / rstan that are on CRAN currently. For example, when you run

example(plot_prediction, package = "walker")

with the provisional StanHeaders, you get an exception

Exception: gamma_lpdf: Shape parameter is 0, but must be positive finite!  (in 'model_walker_lm' at line 195)

that prevents the Markov chains from initializing. If you look at the list of data being passed to your Stan program,

List of 28
 $ k_fixed        : num 1
 $ k_rw1          : num 0
 $ k_rw2          : num 1
 $ m              : num 2
 $ k              : num 1
 $ n              : int 50
 $ n_lfo          : int 50
 $ xreg_fixed     : num [1:50, 1] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:50] "1" "2" "3" "4" ...
  .. ..$ : chr "(Intercept)"
  ..- attr(*, "assign")= int 0
 $ xreg_rw        : num [1, 1:50] 2.201 0.98 1.345 1.014 0.628 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr "x"
  .. ..$ : chr [1:50] "1" "2" "3" "4" ...
 $ y              : Named num [1:50] 4.07 3.31 3.28 2.95 2.59 ...
  ..- attr(*, "names")= chr [1:50] "1" "2" "3" "4" ...
 $ y_miss         : int [1:50] 0 0 0 0 0 0 0 0 0 0 ...
 $ sigma_y_shape  : num 2
 $ sigma_y_rate   : num 0.01
 $ beta_fixed_mean: num 0
 $ beta_fixed_sd  : num 10
 $ beta_rw1_mean  : num 0
 $ beta_rw1_sd    : num 0
 $ beta_rw2_mean  : num 0
 $ beta_rw2_sd    : num 10
 $ sigma_rw1_shape: num 0
 $ sigma_rw1_rate : num 0
 $ sigma_rw2_shape: num 2
 $ sigma_rw2_rate : num 1e-04
 $ nu_mean        : num 0
 $ nu_sd          : num 10
 $ gamma_y        : num [1:50] 1 1 1 1 1 1 1 1 1 1 ...
 $ gamma_rw1      : num[0 , 1:50] 
 $ gamma_rw2      : num [1, 1:50] 1 1 1 1 1 1 1 1 1 1 ...

indeed both sigma_rw1_shape and sigma_rw1_rate are zero, which is not a valid gamma distribution. If you want an improper uniform prior over the positive real numbers (which is not a very good idea), then just omit the prior for a parameter that has been declared with lower = 0. But I think this is more likely a mistake, albeit one that Stan should have caught many versions ago.

We really need you to upload a new walker to CRAN that fixes these issues, but before you do, let me know so that I can test it with the provisional StanHeaders. Thanks!

[helske]

The reason there are Makevars in walker is that walker contains other Stan-independent C++ codes using RcppArmadillo as well (predict.cpp) which need PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) in the Makevars (see stan-dev/rstantools#13). Strange that you can compile the package without it?

Regarding the error in the example, yes sigma_rw1_shape is zero, but as in this example k_rw1 is 0, on line
real<lower=0> sigma_rw1[k_rw1];
I'm declaring a zero-length variable which so far has made line
sigma_rw1 ~ gamma(sigma_rw1_shape, sigma_rw1_rate);
"empty". This is how things have worked so far. I guess Stan now makes some additional checks, I think this could be circumvented by just setting the default "null prior" values to 1 instead of zero.

[helske]

I took a very quick stab on this:

The Makevars files are no longer modified manually, instead, I added a line to configure scripts that append the Makevars created by rstan_config with additional flags.

The prior parameters for zero-length gamma-distributed variables are now positive so in case there is a check regarding the parameters of gamma(sigma_rw1_shape, sigma_rw1_rate); etc. even though the variable on the LHS is length zero, there shouldn't be an error. Actually, now that I think of it, I wonder why I haven't just declared sigma_rw1_shape as real<lower=0>sigma_rw1_shape[k_rw1 > 0] etc. which probably should work as well and be cleaner?...

[helske]

Ok, my idea above didn't make sense (messes dimension checks), but I added checks to ensure that Stan never gets a zero value for parameters with positivity constraint.

@bgoodri Let me know if there are still issues (or new issues arise).