Commandline parameters override array configuration options from file

[askalski85]

Hey folks,
I stumbled upon an issue which I can't address directly. It seems to not be a bug but rather a design decision and I am able to work around it in a slightly inconvenient way.

Background information

feature: remap

My lychee use case relies on testing multiple products documentation. I am using multiple re-mappings as products locally cannot be stored in a single folder. So I am remapping from https urls product A to folder file:///A, product B to file:///B etc. Using the base parameter does not work for me.

I am storing my configuration in my lychee.toml file and I would like to append a few re-mappings from command-line. Since command-line parameters override array configurations from the lychee.toml, I am forced to provide all re-mappings from the command-line.

Other idea which could potentially sort my problem in a direct way and keep all the configs in one place - which is preferred - if lychee could expand environment/template configurations in the lychee.toml file.

Questions

1. It would be nice if I could append existing lychee.toml configuration options for array type properties.
2. It would be nice if lychee could expand templated strings in the lychee.toml file - either from just environment variables like ${{ env.MY_ENV_VAR }}, or custom template variables passed from command-line.

Summary

Please share your thoughts, appreciated.

[mre]

Heyo @askalski85,

I understand your use-case.

Regarding 1: what if a user wants to indeed completely overwrite a setting through the command line? E.g. what if the --remaps I set on the command line should be the only valid ones? In this case, I would have to remember what the overwrite rules are and which variables get overwritten and which ones don't. 😕 It gets complicated pretty quickly.

Regarding 2:

Environment variable substitution in TOML itself was ruled out in 2014 (toml-lang/toml#255) and again in 2017 (toml-lang/toml#496).

The next option would be the toml crate itself, which we use for parsing the config file.
It is not supported there, either. Probably because it's outside the scope of the project and out of fear of vulnerabilities (e.g. shell metacharacter exploits)

I think the easiest would be to substitute the variables yourself with something like envsubst:

# 1. Make sure the environment variables are set first
# 2. Create a lychee.toml.tmpl with the env vars
# 3. To render the lychee config, call
envsubst < lychee.toml.tmpl > lychee.toml

So this could be a workaround for 2. Maybe that works for you?
Maybe you want to contribute to the lychee documentation at https://github.com/lycheeverse/lycheeverse.github.io and add a section about variable substitution if that works? That would greatly help future users with the same need.

Thanks. 😊

[askalski85]

I ended up generating my configuration files as you recommended.