Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom URI based loading of sources extension idea #175

Open
agentgt opened this issue Sep 20, 2024 · 1 comment · May be fixed by #176
Open

Custom URI based loading of sources extension idea #175

agentgt opened this issue Sep 20, 2024 · 1 comment · May be fixed by #176
Milestone

Comments

@agentgt
Copy link

agentgt commented Sep 20, 2024

So I still have not fully replaced our home grown config with avaje-config because of one feature that I can't decide whether to just work around or have it implemented in avaje-config.

What we have in our system that Avaje does not is we have a

  • parser config extension just like Avaje generally based on the file name resource path extension.
  • a filter properties extension - not needed really
  • loading flags which I talked about - not needed really
  • URI schema based custom loading - used a lot!

Basically allow custom loading of resources using URIs. This may already be built with the resource loading SPI but I don't think so.

What I mean by that is we do things like using load.properties as an example:

load.properties=classpath:/somepath.properties env:///MY_PROJECT_PREFIX_DATABASE_/DATABASE_/ gcloud.meta:///

I think the classpath one is obvious and builtin but the env one is builtin for us but I would implement as an extension.

What the environment URI one does is find all environment variables starting with MY_PROJECT_PREFIX_DATABASE_ and replace that starting with DATABASE_. (it does occur to me that perhaps my real use case need is something that just transforms key names but let us just work through this more abstract feature).

Basically internally we have a Map of URI schema to Config loader which is SPI discovered.

// ignore names... you know the bikeshed avoidance drill

interface URILoadingConfigServiceProvider extends ConfigExtension {
  String getSchema();
  Config load(URI uri); // I don't know the exact avaje-config types but you get the idea.
}

Obviously no schema found and it does whatever the defaults are. I realize this adds some more bloat but I think it is not that bad given we have now consolidated service loading to a single call (thank you @SentryMan ).

@agentgt
Copy link
Author

agentgt commented Sep 20, 2024

So my discussion with @SentryMan does not get lost in the abyss that is discord:

[13:52 - 13:58] agentgt:

to be honest it is the killer feature of our in house config system
basically the loading of resources and the configuration of the loading itself is URI driven
so you can use you know like URI query parameters etc
And URI are designed for one line.
Actually rainbowgum uses this as well just far less obvious
The only tricky thing is deciding whether or not you want to pass in some sort of parser lookup
Config load(URI uri, ParserLookupByExtensionOrMimeEtc thingy)
otherwise the URI SPI just has to handle it all which maybe fine.
The only bad thing is documenting URI format and parameters is less obvious than javadoc-ing some builder or getter/setter pojo

[13:58] SentryMan:

How does one get all the env variables with a certain prefix

[13:59 - 14:04] agentgt:

System.getenv gives you map.
its a little messed up because snaphop-kvs basically passes an Environment thing to SPI calls so that System.getenv can be unit tested
ah fuck I said my company's name
oh well
That is System.getenv() returns a map
System properties also has similar
Basically in our system it looks more like Config load(URI uri, ParserLookupByExtensionOrMimeEtc thingy, Environment env) but avaje-config probably does not need that complexity.
Environment has shit like the classloader and facades for system properties / env, system clock etc. Its basically just to ease unit testing.

[14:08 - 14:18] agentgt:

When you do your load.properties split to list if its not a URI or a URI missing schema than you do the default loading mechanism otherwise you go check in your URI schema registry of that SPI
Another awesome one we have is stdin... probably less useful for cloud stuff but useful for CLI
cat some.properties | java -jar some.jar
some.jar has load.properties=stdin:///
Its actually pretty useful for password sensitive stuff
An analog is: https://docs.docker.com/reference/cli/docker/login/#password-stdin

cat ~/my_password.txt | docker login --username foo --password-stdin
I think if stdin is not piped anything it won't block if you are worried about that

If you are wondering what format that stdin plugin parses that is why the parser thing is passed. load.properties=stdin:///?mimeType=text/json // I can't remember what we actually use but you get the idea.
I can't remeber because we basically always use jul properties
the only annoything btw is technically I think you are supposed to escape the : in jul properties but I haven't found it to be a problem.
likewise for =
java util properties is forgiving I guess

[14:22 - 14:26] agentgt:

I'm looking at the URI schemas we support and its a lot. So now I'm really confident if you implement this it would be a massive help for me to transition to avaje-config

CLASSPATH,
JAR,
HTTP,
HTTPS,
STDIN,
CMD,
ENV,
FILE,

that is just the builtin
Oh guess what CMD does ... e.g. cmd:///
command like arguments
For example I think to emulate -D like Maven where its passed after the .jar you can do

load.properties=cmd:/-D/

Sorry for poor formatting!

@SentryMan SentryMan added this to the 4.1 milestone Sep 25, 2024
@SentryMan SentryMan linked a pull request Oct 9, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants