Skip to content

[DBMON-6334] Add ClickHouse unit tests to track all config default value changes#23057

Open
sangeetashivaji wants to merge 2 commits intomasterfrom
sangeeta.shivajirao/test_config_defaults
Open

[DBMON-6334] Add ClickHouse unit tests to track all config default value changes#23057
sangeetashivaji wants to merge 2 commits intomasterfrom
sangeeta.shivajirao/test_config_defaults

Conversation

@sangeetashivaji
Copy link
Contributor

@sangeetashivaji sangeetashivaji commented Mar 25, 2026

What does this PR do?

The goal of this PR is to ensure our default config values remain the same. We're now explicitly asserting against all default rendered config values from InstanceConfig. Any changes to our hardcoded expected defaults will trigger a test failure with a clear message as to why it's failing and steps to take to mitigate. Similarly if new config values are added and they aren't asserted against in this test it will trigger a failure and require fixing up.

Example failure message when a new testConfig value is added but wasn't updated in the test
image

comparing the defaults from the previous inline config here vs the new ClickHouse DBM support PR
image

Motivation

The goal of this change is to ensure our config default values don't change unexpectedly or accidentally overtime to avoid regressions.

Tests:
Config defaults tests
image

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@github-actions
Copy link
Contributor

github-actions bot commented Mar 25, 2026

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

@sangeetashivaji sangeetashivaji force-pushed the sangeeta.shivajirao/test_config_defaults branch from 0a64704 to 3927649 Compare March 25, 2026 21:30
@sangeetashivaji sangeetashivaji changed the title [DRAFT] Add config tests for new ClickHouse config updates [DRAFT] Add ClickHouse unit tests to track all config default value changes Mar 25, 2026
@sangeetashivaji sangeetashivaji added the qa/skip-qa Automatically skip this PR for the next QA label Mar 25, 2026
@sangeetashivaji sangeetashivaji marked this pull request as draft March 25, 2026 21:33
@codecov
Copy link

codecov bot commented Mar 25, 2026

Codecov Report

❌ Patch coverage is 49.15254% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.81%. Comparing base (caad268) to head (ab4f94a).
⚠️ Report is 11 commits behind head on master.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sangeetashivaji sangeetashivaji changed the title [DRAFT] Add ClickHouse unit tests to track all config default value changes [DBMON-6334] Add ClickHouse unit tests to track all config default value changes Mar 26, 2026
@sangeetashivaji sangeetashivaji marked this pull request as ready for review March 26, 2026 15:15
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39276492a5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +174 to +176
for key, expected_value in expected_dict.items():
try:
actual_value = getattr(actual_obj, key)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Detect untracked keys in nested defaults

The nested comparison only loops over expected_dict.items() and never checks for extra attributes present on the actual nested model, so new fields added under objects like query_metrics or query_samples can slip in without failing this test. In that scenario, missing_from_expected still passes (top-level field unchanged) and _compare_nested_object also passes, which breaks the stated guarantee that every default change is explicitly tracked.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: added nested fields check too
Example: added check for nested fields in Query metrics

'query_metrics': {                                                                                                                                                                                                  
      'enabled': True,                                                                                                                                                                                                
      'collection_interval': 10,                                                                                                                                                                                      
      'run_sync': False,                                    
      'full_statement_text_cache_max_size': 10000,                                                                                                                                                                    
      'full_statement_text_samples_per_hour_per_query': 1,                                                                                                                                                            
  },    

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant