Add TemplateFS for templated file handling in migrations

[BupycHuk]

• Introduced a new TemplateFS struct to wrap embed.FS and apply Go text/template processing to file content.
• Updated the runMigrations function to utilize TemplateFS for reading migration files with templating.
• Added unit tests for TemplateFS functionality, including conditional templating and error handling.
• Created sample SQL template files for testing purposes, demonstrating both valid and invalid template syntax.

PMM-0

Link to the Feature Build: SUBMODULES-0

If this PR adds or removes or alters one or more API endpoints, please review and add or update the relevant API documents as well:

• API Docs updated

If this PR is related to some other PRs in this or other repositories, please provide links to those PRs:

• Links to related pull requests (optional).

TemplateFS - Embedded Filesystem with Go Template Support

Overview

TemplateFS is a custom fs.FS implementation that wraps an embed.FS and applies Go text/template processing to file content during reads. This allows you to embed template files and dynamically substitute variables at runtime.

Key Features

• Simple Interface: Implements same interfaces as embed.FS (fs.FS, fs.ReadDirFS, fs.ReadFileFS)
• Template Processing: Applies Go text/template to file content via ReadFile()
• Original File Access: Open() returns original files without templating
• Standard Library Compatible: Works with fs.Sub(), fs.Glob() and other standard functions
• Minimal Overhead: Only processes templates when explicitly reading file content

Usage

Basic Example

//go:embed migrations/*.sql
var migrationFiles embed.FS

// Define simple template data that will be used for all files
templateData := map[string]any{
    "DatabaseName": "myapp",
}

// Create TemplateFS
tfs := templatefs.NewTemplateFS(migrationFiles, templateData)

// Read file with templating applied
content, err := tfs.ReadFile("migrations/001_create_table.sql")
if err != nil {
    log.Fatal(err)
}

// Content now has template variables substituted
fmt.Println(string(content))

Integration with golang-migrate

func runMigrationsWithTemplating(dsn string) error {
    // Create TemplateFS with simple template data
    templateData := map[string]any{
        "DatabaseName": "pmm",
    }
    tfs := templatefs.NewTemplateFS(migrationFiles, templateData)
    
    // Use TemplateFS directly with golang-migrate
    d, err := iofs.New(tfs, "migrations/sql")
    if err != nil {
        return err
    }
    
    m, err := migrate.NewWithSourceInstance("iofs", d, dsn)
    if err != nil {
        return err
    }
    
    return m.Up()
}

Template Files

Template files use standard Go text/template syntax:

-- 001_create_table.sql
CREATE TABLE {{.DatabaseName}}.metrics (
    id BIGINT PRIMARY KEY,
    timestamp DateTime,
    value Float64,
    created_at TIMESTAMP DEFAULT NOW()
);

API Reference

TemplateFS

type TemplateFS struct {
    EmbedFS embed.FS                    // Underlying embedded filesystem
    Data    map[string]any              // Template data used for all files
}

Methods

• NewTemplateFS(embedFS embed.FS, data map[string]any) *TemplateFS

  • Creates a new TemplateFS instance

• Open(name string) (fs.File, error)

  • Returns original file without templating (for compatibility)

• ReadFile(name string) ([]byte, error)

  • Main method: Returns file content with templating applied

• ReadDir(name string) ([]fs.DirEntry, error)

  • Lists directory contents (delegates to embed.FS)

Template Data Usage

The same template data is used for all files in the filesystem. This makes it simple to have consistent variables across all your template files:

• All files use the same Data map for template variable substitution
• Variables like {{.DatabaseName}} are available in all template files
• If Data is nil, template processing is skipped and original content is returned

Error Handling

• Template parsing errors are silently ignored (original content returned)
• Template execution errors are silently ignored (original content returned)
• File system errors (file not found, etc.) are propagated normally

This ensures that TemplateFS is backwards compatible with regular embedded files.

Example Usage in PMM

In the PMM project, TemplateFS is used in the QAN API for database migrations:

func runMigrations(dsn string) error {
    // Create TemplateFS with simple template data
    templateData := map[string]any{
        "DatabaseName": "pmm",
    }
    tfs := templatefs.NewTemplateFS(migrationFS, templateData)
    
    // Use TemplateFS directly with golang-migrate
    d, err := iofs.New(tfs, "migrations/sql")
    if err != nil {
        return err
    }
    
    m, err := migrate.NewWithSourceInstance("iofs", d, dsn)
    if err != nil {
        return err
    }
    
    return m.Up()
}

This allows migration files to use template variables like {{.DatabaseName}} for dynamic database naming while maintaining full compatibility with existing non-templated migrations.