Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Make RawConfig serializable #383

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Creative-Difficulty
Copy link

@Creative-Difficulty Creative-Difficulty commented Jul 16, 2024

Related to #382.

I have implemented custom serializers based on the deserializers already in the code.

@Creative-Difficulty
Copy link
Author

The test i added tests both serialization and deserialization, since the RawConfig can't be constructed programmatically. Should i rename it/the full_deserialize test or remove the full_deserialize test, since the new test incorporates its functionality?

@Creative-Difficulty
Copy link
Author

@estk

@bconn98
Copy link
Collaborator

bconn98 commented Jul 17, 2024

I would keep the tests, but maybe store the config as a static that all the tests can use instead of copy/pasting it

@Creative-Difficulty
Copy link
Author

@bconn98 So the actual implementation is correct?

@Creative-Difficulty
Copy link
Author

@bconn98 cfg is now a constant. Would you mind checking my implementation for correctness, as I didn't write any custom serializers, however the deserialize implementation uses a custom deserializer?

@Creative-Difficulty
Copy link
Author

@bconn98

@bconn98
Copy link
Collaborator

bconn98 commented Aug 2, 2024

@Creative-Difficulty So this is not good in it's current state. Due to how the library stores things, the default serialized configs can't be parsed by the library. Here is an example of printing out the output generated in your test.

refresh_rate:
  secs: 60
  nanos: 0
root:
  level: INFO
  appenders:
  - console
appenders:
  baz:
    kind: file
    filters: []
    config:
      encoder:
        pattern: '%m'
      path: /tmp/baz.log
  console:
    kind: console
    filters:
    - kind: threshold
      config:
        level: debug
    config: {}
loggers:
  foo::bar::baz:
    level: WARN
    appenders:
    - baz
    additive: false

Although we don't want to add PartialEq to everything in the library. For testing, you can add partial equal to RawConfig and other required components. Then try to parse you serialized config and compare the RawConfig from our static against the RawConfig deserialized from your serialized config.

@Creative-Difficulty
Copy link
Author

Creative-Difficulty commented Aug 4, 2024

@bconn98 I have a question: The humantime crate used for serializing/deserializing the refresh_rate option in RawConfig serializes 1 minute (as a Rust Duration) into the the string 1m. This means that de- and then reserializing the same config modifies it. I could hardcode m to be replaced with minute (or minutes when applicable) and so on with hours and days. Is this a cocern? Should I hardcode a translation as to not break the current examples?

@bconn98
Copy link
Collaborator

bconn98 commented Aug 5, 2024

@Creative-Difficulty sounds like a translation is warranted here yup

@Creative-Difficulty
Copy link
Author

@bconn98 In the latest commit serialization of filters and appenders is fixed. Currently 54 tests are passing with 1 failed and 1 ignored. The current output (of config::raw::test::full_serialize) is:

refresh_rate: 1m
root:
  level: INFO
  appenders:
  - console
appenders:
  baz:
    encoder:
      pattern: '%m'
    filters: '[]'
    kind: file
    path: /tmp/baz.log
  console:
    filters:
    - kind: threshold
      level: debug
    kind: console
loggers:
  foo::bar::baz:
    level: WARN
    appenders:
    - baz
    additive: false

However there are two issues:

  1. Time formatting: Hardcoding a translation is a possibility, however in that case 60 seconds from the example(s) would be converted to 1m by the humantime crate, so the example config (const CFG) would still be modified if de- and then reserialized. In my opinion this defeats the whole point of trying not to modify the example configs when de- and reserializing them, so translations would be unnecessary from my point of view.

  2. Indentation: The serde_yaml crate auto-indents its output with 2 spaces. Some issues (Yaml output indentation dtolnay/serde-yaml#337, Allow the indentation character to be customized dtolnay/serde-yaml#343) have already been opened, however I don't see anything being fixed/added soon since the repo is archived.

How do I proceed?

Thank you in advance!

@bconn98
Copy link
Collaborator

bconn98 commented Aug 10, 2024

For #1, I think it's unfortunate but likely the best we can do with the existing crate backend. It's something we should include in documentation though. For #2, I'm not worried about the spacing as long as it's valid yaml.

@Creative-Difficulty
Copy link
Author

@bconn98 So is the PR ready?

@Creative-Difficulty
Copy link
Author

Creative-Difficulty commented Sep 3, 2024

@bconn98 @estk @sfackler @demonov Is this PR ready for review/merge?

@bconn98
Copy link
Collaborator

bconn98 commented Sep 4, 2024 via email

@Creative-Difficulty Creative-Difficulty changed the title feat: Make RawConfig serializable fix: Make RawConfig serializable Sep 16, 2024
@Creative-Difficulty Creative-Difficulty changed the title fix: Make RawConfig serializable feat: Make RawConfig serializable Sep 16, 2024
@izolyomi
Copy link

izolyomi commented Dec 3, 2024

Thank you for the efforts. We'd also like to see this feature merged if possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants