Simplify testing infrastructure by removing DM_TEST_SRC and implementing template-based test generation

[Copilot]

This PR modernizes the testing infrastructure by removing the global DM_TEST_SRC environment variable dependency and implementing a template-based system for database-specific test duplication.

Problem

The previous testing system relied heavily on the DM_TEST_SRC environment variable to switch between different database backends (postgres, maria, mssql, sqlite, duckdb). This created several issues:

• Tests were tightly coupled to environment configuration
• Database-specific test files contained nearly identical duplicated code
• Maintaining consistency across database variants was error-prone
• CI workflows were complex with matrix configurations setting DM_TEST_SRC

Solution

Template-Based Test Generation

Created a new system where database-specific tests are generated from a single template:

• Template file: tests/testthat/template-db-tests.R contains test logic with {{DATABASE}} placeholders
• Generation script: generate-db-tests.sh replaces placeholders and creates individual test files
• Self-contained tests: Each generated file sets up its own database connection locally
• Synchronization: All database tests stay in sync because they're generated from the same source

Infrastructure Modernization

• Removed DM_TEST_SRC dependency: Tests now default to data frame mode and database tests are self-contained
• Updated CI workflows: Changed from DM_TEST_SRC to DM_SETUP_DB for cleaner database service configuration
• Enhanced build system: Added make generate-db-tests target and removed environment variable dependencies
• Comprehensive documentation: Updated all relevant docs and added detailed README for the new system

Benefits

• Reduced duplication: Single template maintains all database test variants
• Improved maintainability: Changes only need to be made in one place
• Simplified CI: Database services set up independently of test execution
• Better developer experience: Clear tooling and documentation for test management
• Backward compatibility: Existing skip patterns continue to work

Usage

# Generate database test files from template
make generate-db-tests

# Run tests (automatically uses appropriate backends)
R -e 'testthat::test_local()'

# Run specific database tests
R -e 'testthat::test_local(filter = "postgres")'

The template system ensures that database-specific tests remain synchronized while eliminating the complexity of the previous environment variable-based approach.

Fixes #1981.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.