Fix log level: CLI phase suppression, case-insensitive "default" key, dynamic runtime log level

[Copilot]

Why make this change?

Implements and refines the log level improvements from PR #3203, addressing all open review comments:

1. dab start --LogLevel Warning still emitted CLI-phase Information messages because CustomConsoleLogger had a hardcoded LogLevel.Information minimum and the logger factory was created before CLI args were parsed.
2. Config-defined log level key "Default" (capital D) was silently ignored — TryGetValue("default") is case-sensitive.
3. Log filter segment comparison in IsLoggerFilterValid was case-sensitive, rejecting valid mixed-case namespace filters.
4. No mechanism to defer log output until after the runtime config's log level was known, causing early startup messages to bypass the configured level.

What is this change?

New: DynamicLogLevelProvider (src/Service/Telemetry/)

• Singleton holding the current minimum log level; initialised from CLI args, then optionally updated from RuntimeConfig unless the CLI already overrode it.
• Log filters in Program.GetLoggerFactoryForLogLevel and ConfigureLogging delegate to LogLevelProvider.ShouldLog() instead of capturing a static level at startup.

New: StartupLogBuffer (src/Config/)

• Thread-safe queue that captures log entries before the DI container has a real ILogger.
• FileSystemRuntimeConfigLoader writes to the buffer during early startup; Startup.Configure calls FlushLogBuffer() once the proper logger is resolved.

CLI phase respects --LogLevel

• Cli/Program.Main calls new PreParseLogLevel(args) before CommandLine.Parser runs, wiring the result into CustomLoggerProvider(minimumLogLevel).
• CustomConsoleLogger._minimumLogLevel is now constructor-injected instead of hardcoded to Information.

Bug fixes

• RuntimeConfig.GetConfiguredLogLevel: StartsWith → OrdinalIgnoreCase; "default" key lookup uses FirstOrDefault with OrdinalIgnoreCase (was TryGetValue("default"), exact match only).
• RuntimeConfigValidator.IsLoggerFilterValid: segment comparison uses OrdinalIgnoreCase.
• FileSystemRuntimeConfigLoader: all Console.WriteLine calls replaced with private LogInformation/LogWarning/LogError helpers that route _logger → buffer → Console; eliminated duplicate log entry in hot-reload path.
• Program.StartEngine: removed Console.WriteLine("Starting the runtime engine...") (suppressed at Warning+ but was checked by tests regardless).

Test refactoring

• TestEngineStartUpWithVerboseAndLogLevelOptions split into two [DataTestMethod] groups:
  • Trace/Debug/Information: asserts CLI output lines are present.
  • Warning/Error/Critical/None: uses process.WaitForExit(1000) + Assert.IsFalse(exitedWithinTimeout) — fixes the race condition from Assert.IsFalse(process.HasExited).
• ValidStringLogLevelFilters gains a "Default" (capital D) data row.

How was this tested?

• Integration Tests
• Unit Tests
  • StartupLogBufferTests (4): replay order, null-logger discard, idempotent flush, empty buffer
  • DynamicLogLevelProviderTests (6): SetInitialLogLevel, UpdateFromRuntimeConfig with/without CLI override, ShouldLog boundary cases
  • GetConfiguredLogLevelTests (3): "default" / "Default" / "DEFAULT" variants, specific-filter-over-default priority
  • ProgramPreParseLogLevelTests (22): all numeric and named log levels including mixed-case, edge cases (absent flag, invalid value, out-of-range, flag-with-no-value)

Sample Request(s)

# CLI-phase messages now suppressed when --LogLevel Warning is passed
dab start --config dab-config.json --LogLevel Warning
# No "Information: Microsoft.DataApiBuilder v..." output

# Config file with title-case "Default" key now works correctly
# runtime.telemetry.log-level: { "Default": "Warning" }
dab start --config dab-config.json

Original prompt

review this PR. also check the open review comments if any. consolidate with your review comments and generate the details here. implement the fixes here. ensure test case coverage. the functionality should work end to end. refactor the code and keep it modular including test cases by merging similar tests into datatestmethod.

The user has attached the following file paths as relevant context:

• .github\copilot-instructions.md

Created from VS Code.

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.