-
Notifications
You must be signed in to change notification settings - Fork 47
Inference model configuration setup
Need to add multiTimeReduce and hospitalization interventions
This documentation describes the new YAML configuration file options that may be used when performing inference on model runs. As compared to previous model releases, there are additions to the seeding and interventions sections, the outcomes section replaces the hospitalization section, and the filtering section added to the file.
Importantly, we now name our pipeline modules: seeding, seir, hospitalization and this becomes relevant to some of the new filtering specifications.
Models may be calibrated to any available time series data that is also an outcome of the model (COVID-19 confirmed cases, deaths, hospitalization or ICU admissions, hospital or ICU occupancy, and ventilator use). Our typical usage has calibrated the model to deaths, confirmed cases, or both. We can also perform inference on intervention effectiveness, county-specific baseline R0, and the risk of specific health outcomes.
We describe these options below and present default values in the example configuration sections.
The model can perform inference on the seeding date and initial number of seeding infections in each geoid. An example of this new config section is:
seeding:
method: FolderDraw
seeding_file_type: seed
folder_path: importation/minimal/
lambda_file: data/minimal/seeding.csv
perturbation_sd: 3
| Config Item | Required? | Type/Format | Description |
|---|---|---|---|
| method | required | "FolderDraw" | |
| seeding_file_type | required for FolderDraw | "seed" or "impa" | indicates which seeding file type the SEIR model will look for, "seed", which is generated from create_seeding.R, or "impa", which refers to importation |
| folder_path | required | path to folder where importation inference files will be saved | |
| lambda_file | required | path to seeding file | |
| perturbation_sd | required | standard deviation for the proposal value of the seeding date, in number of days |
The method for determining the proposal distribution for the seeding amount is hard-coded in the inference package (R/pkgs/inference/R/functions/perturb_seeding.R). It is pertubed with a normal distribution where the mean of the distribution 10 times the number of confirmed cases on a given date and the standard deviation is 1.
The model can perform inference on the effectiveness of interventions as long as there is at least some calibration health outcome data that overlaps with the intervention period. For example, if calibrating to deaths, there should be data from time points where it would be possible to observe deaths from infections that occurred during the intervention period (e.g., assuming 10-18 day delay between infection and death, on average).
An example configuration file where inference is performed on scenario planning interventions is as follows:
interventions:
scenarios:
- Scenario1
settings:
local_variance:
template: ReduceR0
value:
distribution: truncnorm
mean: 0
sd: .1
a: -1
b: 1
perturbation:
distribution: truncnorm
mean: 0
sd: .1
a: -1
b: 1
stayhome:
template: ReduceR0
period_start_date: 2020-04-04
period_end_date: 2020-04-30
value:
distribution: truncnorm
mean: 0.6
sd: 0.3
a: 0
b: 0.9
perturbation:
distribution: truncnorm
mean: 0
sd: .1
a: -1
b: 1
Scenario1:
template: Stacked
scenarios:
- local_variance
- stayhome
This configuration allows us to infer geoid-level baseline R0 estimates by adding a local_variance intervention. The baseline geoid-specific R0 estimate may be calculated as
Interventions may be specified in the same way as before, or with an added perturbation section that indicates that inference should be performed on a given intervention's effectiveness. As previously, interventions with perturbations may be specified for all modeled locations or for explicit affected_geoids. In this setup, both the prior distribution and the range of the support of the final inferred value are specified by the value section. In the configuration above, the inference algorithm will search 0 to 0.9 for all geoids to estimate the effectiveness of the stayhome intervention period. The prior distribution on intervention effectiveness follows a truncated normal distribution with a mean of 0.6 and a standard deviation of 0.3. The perturbation section specifies the perturbation/step size between the previously-accepted values and the next proposal value.
| Item | Required? | Type/Format |
|---|---|---|
| template | Required | "ReduceR0" or "Stacked" |
| period_start_date | optional for ReduceR0 | date between global start_date and end_date; default is global start_date
|
| period_end_date | optional for ReduceR0 | date between global start_date and end_date; default is global end_date
|
| value | required for ReduceR0 | specifies both the prior distribution and range of support for the final inferred values |
| perturbation | optional for ReduceR0 | this option indicates whether inference will be performed on this setting and how the proposal value will be identified from the last accepted value |
| affected_geoids | optional for ReduceR0 | list of geoids, which must be in geodata |
This section is now structured more like the interventions section of the config, in that it has scenarios and settings. We envision that separate scenarios will be specified for each IFR assumption.
outcomes:
method: delayframe
param_from_file: TRUE
param_place_file: "usa-geoid-params-output.parquet" ## ../../Outcomes/data/usa-geoid-params-output.parquet
scenarios:
- med
settings:
med:
incidH:
source: incidI
probability:
value:
distribution: fixed
value: .035
delay:
value:
distribution: fixed
value: 7
duration:
value:
distribution: fixed
value: 7
name: hosp_curr
incidD:
source: incidI
probability:
value:
distribution: fixed
value: .01
delay:
value:
distribution: fixed
value: 20
incidICU:
source: incidH
probability:
value:
distribution: fixed
value: 0.167
delay:
value:
distribution: fixed
value: 3
duration:
value:
distribution: fixed
value: 8
incidVent:
source: incidICU
probability:
value:
distribution: fixed
value: 0.463
delay:
value:
distribution: fixed
value: 1
duration:
value:
distribution: fixed
value: 7
incidC:
source: incidI
probability:
value:
distribution: truncnorm
mean: .1
sd: .1
a: 0
b: 10
perturbation:
distribution: truncnorm
mean: 0
sd: .1
a: -1
b: 1
delay:
value:
distribution: fixed
value: 7
| Item | Required? | Type/Format |
|---|---|---|
| method | required | "delayframe" |
| param_from_file | required | if TRUE, will look for param_place_file |
| param_place_file | optional | path to geoid-params parquet file, which indicates location specific risk values. Values in this file will override values in the config if there is overlap. |
| scenarios | required | user-defined scenario name |
| settings | required | See details below |
The settings for each scenario correspond to a set of different health outcome risks, most often just differences in the probability of death given infection (Pr(incidD|incidI)) and the probability of hospitalization given infection (Pr(incidH|incidI)). Each health outcome risk is referenced in relation to the outcome indicated in source. For example, the probability and delay in becoming a confirmed case (incidC) is most likely to be indexed off of the number and timing of infection (incidI).
Importantly, we note that incidI is automatically defined from the SEIR transmission model outputs, while the other compartment sources must be defined in the config before they are used.
Users must specific two metrics for each health outcome, probability and delay, while a duration is optional (e.g., duration of time spent in the hospital). It is also optional to specify a perturbation section (similar to perturbations specified in the NPI section) for a given health outcome and metric. If you want to perform inference (i.e., if perturbation is specified) on a given metric, that metric must be specified as a distribution (i.e., not fixed) and the range of support for the distribution represents the range of parameter space explored in the inference.
| Item | Required? | Type/Format |
|---|---|---|
| (health outcome metric) | required | "incidH", "incidD", "incidICU", "incidVent", "incidC", corresponding to variable names |
| source | required | name of health outcome metric that is used as the reference point |
| probability | required | health outcome risk |
| probability::value | required | specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution |
| probability::perturbation | optional | inference settings for the probability metric |
| delay | required | time delay between source and the specified health outcome |
| delay::value | required | specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution |
| delay::perturbation | optional | inference settings for the time delay metric (coming soon) |
| duration | optional | duration that health outcome status endures |
| duration::value | required | specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution |
| duration::perturbation | optional | inference settings for the duration metric (coming soon) |
This section configures the settings for the inference algorithm. The below example shows the settings for some typical default settings, where the model is calibrated to the weekly incident deaths and weekly incident confirmed cases for each geoid. Statistics, hierarchical_stats_geo, and priors each have scenario names (e.g., sum_deaths, local_var_hierarchy, and local_var_prior, respectively).
filtering:
simulations_per_slot: 350
do_filtering: TRUE
data_path: data/observed_data.csv
likelihood_directory: importation/likelihood/
statistics:
sum_deaths:
name: sum_deaths
aggregator: sum ## function applied over the period
period: "1 weeks"
sim_var: incidD
data_var: death_incid
remove_na: TRUE
add_one: FALSE
likelihood:
dist: sqrtnorm
param: [.1]
sum_confirmed:
name: sum_confirmed
aggregator: sum
period: "1 weeks"
sim_var: incidC
data_var: confirmed_incid
remove_na: TRUE
add_one: FALSE
likelihood:
dist: sqrtnorm
param: [.2]
hierarchical_stats_geo:
local_var_hierarchy:
name: local_variance
module: seir
geo_group_col: USPS
transform: none
local_conf:
name: probability_incidI_incidC
module: hospitalization
geo_group_col: USPS
transform: logit
priors:
local_var_prior:
name: local_variance
module: seir
likelihood:
dist: normal
param:
- 0
- 1
With inference model runs, the number of simulations nsimulations refers to the number of final model simulations that will be produced. The filtering$simulations_per_slot setting refers to the number of iterative simulations that will be run in order to produce a single final simulation (i.e., number of simulations in a single MCMC chain).
| Item | Required? | Type/Format |
|---|---|---|
| simulations_per_slot | required | number of iterations in a single MCMC inference chain |
| do_filtering | required | TRUE if inference should be performed |
| data_path | required | file path where observed data are saved |
| likelihood_directory | required | folder path where likelihood evaluations will be stored as the inference algorithm runs |
| statistics | required | specifies which data will be used to calibrate the model. see filtering::statistics for details |
| hierarchical_stats_geo | optional | specifies whether a hierarchical structure should be applied to any inferred parameters. See filtering::hierarchical_stats_geo for details. |
| priors | optional | specifies prior distributions on inferred parameters. See filtering::priors for details |
The statistics specified here are used to calibrate the model to empirical data. If multiple statistics are specified, this inference is performed jointly and they are weighted in the likelihood according to the number of data points and the variance of the proposal distribution.
| Item | Required? | Type/Format |
|---|---|---|
| name | required | name of statistic, user defined |
| aggregator | required | function used to aggregate data over the period, usually sum or mean |
| period | required | duration over which data should be aggregated prior to use in the likelihood, may be specified in any number of days, weeks, months
|
| sim_var | required | column name where model data can be found, from the hospitalization outcomes files |
| data_var | required | column where data can be found in data_path file |
| remove_na | required | logical |
| add_one | required | logical, TRUE if evaluating the log likelihood |
| likelihood::dist | required | distribution of the likelihood |
| likelihood::param | required | parameter value(s) for the likelihood distribution. These differ by distribution so check the code in inference/R/functions.R/logLikStat function. |
The hierarchical settings specified here are used to group the inference of certain parameters together (similar to inference in "hierarchical" or "fixed/group effects" models). For example, users may desire to group all counties in a given state because they are geograhically proximate and impacted by the same statewide policies. The effect should be to make these inferred parameters follow a normal distribution and to observe shrinkage among the variance in these grouped estimates.
| Item | Required? | Type/Format |
|---|---|---|
| scenario name | required | name of hierarchical scenario, user defined |
| name | required | name of the estimated parameter that will be grouped (e.g., the NPI scenario name or a standardized, combined health outcome name like probability_incidI_incidC) |
| module | required | name of the module where this parameter is estimated (important for finding the appropriate files) |
| geo_group_col | required | geodata column name that should be used to group parameter estimation |
| transform | required | type of transform that should be applied to the likelihood: "none" or "logit" |
It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.
| Item | Required? | Type/Format |
|---|---|---|
| scenario name | required | name of prior scenario, user defined |
| name | required | name of NPI scenario or parameter that will have the prior |
| module | required | name of the module where this parameter is estimated |
| likelihood | required | specifies the distribution of the prior |