Site-specific customization options

[mikix]

I've chatted with a few people about this informally, but I wanted to bring this conversation to a broader-but-still-informal context 😄

There looks like there will be the need for some site-specific customization of bits of the Cumulus project. And I wanted to gather thoughts about how that might work.

Some use cases I've heard recently are below. Not every one needs to be a site customization option. These are just brainstorm ideas for what we might want to allow.

Use Case Brainstorming

1. Currently, the core Library study only looks at data post-2016. This is kind of a performance / only-interested-in-recent-data cutoff. That should probably be overrideable.
2. A site might have custom code systems in use (e.g. not standard SNOMED codes, but something else). The ETL has NLP tasks that examine DocRef categories for ED visits - that list of codes/systems might want to allow for site customization (in fact, BCH has some custom codes and systems hardcoded right now in that task). Similarly, Library studies (core and otherwise) examine codes & systems all the time. They might also want to allow flexibility.
3. I'm currently working on a study that holds various data metrics, to look for quality issues. One of those metrics is "do your dates look plausible?" and we are thinking of calling anything before 1950 as "not plausible" - but that feels like something that should be overrideable.
4. Similarly, that data metrics study might want to warn if certain measurements cross a threshold (e.g. "over 5% of your Encounters have broken links to Patients"). And maybe those thresholds should be tweakable by sites.

Plugins

These next couple are more like use cases for plugins, rather than config customization. Which might be out of scope for this specific discussion, but something to keep in mind.

1. Long term, the ETL should not require study tasks to be hard-coded in the upstream repo. It should allow a study to provide its own task code. (This isn't necessarily site-specific - but is customization adjacent.)
2. How we turn HTML templates in clinical notes might need per-site customization. Right now, we use inscriptis to strip the HTML. But what if that isn't sufficient for a site's unusual HTML templates? They might need actual code to transform the HTML correctly. That feels like a plugin kind of thing.

Open Questions

• Anyone got more use cases?
• How much customization do we want to allow? Defaults with site overrides in a config file somewhere make some sense. What about command line or environment variable overrides? Is that a feature or a mistake to allow that flexibility?
• Where does that config live? Should the covid_symptom ETL tasks use the same config file as the covid_symptom Library study? Should Library studies share the same file between them, or should we keep those separate?
• How far down should the config be propagated, as a way of understanding how the data was generated? i.e. should we push it to Athena in some fashion? To the aggregator? Consider both ETL and Library configs here too.
• Currently, Library studies have a study config file in the manifest.toml file. Which is aimed at study authors, not site operators, to be fair. But is a convenient enough place to drop site config that we should throw it in there too? Is toml vs something else desired here?
• Grabbing a config file library vs just writing some config glue ourselves. This consideration can probably only be done once we understand our needs better.

[dogversioning]

This is not a full response, but a use case that this would enable that i'm currently bumping up against: the ability to override the bin size at a project/study level would be helpful for testing.

[dogversioning]

A use case from the covid study: we may want to allow more than one source of coding data (i.e. structured from code systems, freeform match against the text fields) in a study, and let a study runner specify the appropriate means of IDing PCR data.

[mikix]

Small note that for the data-metrics study (still private while it is WIP), I wanted a quick and dirty way to run it in two different "modes".

I ended up just checking an environment variable, for hacking it in quickly.

But Matt and I also talked about how a reasonable path forward for that kind of thing would be a CLI approach like:

cumulus-library --study-option key:value (naming and formatting tbd)

Which could be a wedge into broader customization, including config files.