Add name matching capabilities

Cargo.toml

@@ -56,6 +56,8 @@ libc = "0.2"
 lz4 = "1"
 pglz = "0.1"
 postgres-protocol = "0.6"
+globset = "0.4"
+regex-automata = "0.4"
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
 smallvec = "1"

README.md

@@ -140,28 +140,72 @@ host = "localhost"
 database = "analytics"
 
 # Opt a table out (wins over replicate_all):
-[table."public.audit_log"]
+[table.public.audit_log]
 replicate = false
 
 # Or list explicitly — replicate only what you name:
 [stream]
 replicate_all = false
-[table."public.users"]
+[table.public.users]
 replicate = true
 initial_load = "none"
-target = "users"
+target_table = "users"
 columns = [
     { attnum = 1, target = "id",    type = "UInt64" },
     { attnum = 2, target = "name",  type = "String" },
 ]
 ```
 
+Table blocks take two key levels, `[table.<namespace>.<relname>]`; a name
+carrying a dot or other TOML-special character quotes per key rules, e.g.
+`[table.public."odd.name"]`.
+
 `replicate_all` skips system schemas (`pg_*`, `information_schema`, the
 `[runtime_config]` schema). `attnum` values match `pg_attribute.attnum`
 (1-based) on the source relation; `type` is the CH destination type walshadow
 advertises in the INSERT block. SIGHUP reloads mappings atomically;
 connection params stay boot-only.
 
+Name a set of tables instead of one, with `match`:
+
+```toml
+[table.app."events_*"]              # each name part an anchored pattern
+match = "glob"                      # exact (default) | glob | regex
+replicate = true
+initial_load = "copy"
+
+[table.app."*_audit"]               # a guardrail: excludes even what
+match = "glob"                      # a wider opt-in swept in
+replicate = false
+```
+
+Everything a `config_table` row carries — destination, `replicate`,
+`initial_load` — also takes `match = "glob"` / `"regex"`, which is how you
+scope tables that do not exist yet: under `auto_create` / `replicate_all` a
+relation reaches CH the first time it is seen, before a row naming it could
+arrive. See [plans/config.md](plans/config.md).
+
+Columns take the same treatment. A `columns` entry keys on `attnum` or on
+`name`, never both, and one array holds one kind:
+
+```toml
+[table.app."*"]
+match = "glob"
+replicate = true
+columns = [
+    { name = "*_at", match = "glob", type = "DateTime64(6, 'UTC')" },
+    { name = "legacy_id", target = "id" },   # rename, keep the bridge type
+]
+```
+
+An `attnum` entry pins the projection outright, so it needs `target` + `type`
+and only fits a block naming one relation. A `name` entry states the CH name
+or type of a column the descriptor yields anyway: `target` and `type` are each
+optional, a `match` entry may not set `target` (several columns cannot share
+one CH name), and the whole array pins nothing — scope still comes from
+`replicate` / `auto_create`. `config_column` rows take `match` the same way,
+over the whole `(namespace, relname, attname)` key.
+
 
 ## Building from source
 

plans/config.md

@@ -29,12 +29,8 @@ untouched.
 - `tables` — per-relation destination mapping, keyed `"<namespace>.<relname>"`
 - `namespaces` — per-namespace defaults (`auto_create`, `target_database`,
   `drop_table_strategy`)
-- `columns` — per-column CH-type override from the `config_column` overlay,
-  keyed `"<namespace>.<relname>"` → source attname → CH type expression,
-  WAL-tracked. Consumed by `TablePlan::build` (the batcher's plan cache),
-  which resolves attname→attnum against the descriptor at hand and swaps the
-  column's encode type when the override is wire-compatible (see §column
-  overrides below)
+- `column_rules` — per-column CH names and types from TOML and
+  `config_column`, applied when building mappings, DDL, and encoder plans
 - `drop_table_strategy` — global DROP fallback; per-namespace overrides it
 - `row_budget`, `byte_budget`, `flush_timeout` — emitter batch-seal triggers,
   read live by the batcher per seal decision (ticker re-armed on change)
@@ -82,7 +78,8 @@ left as it stands, since its retention is what a rollback to that server
 reads.
 `target_database`, `soft_delete`, `[stream] replicate_all`, and the
 `[runtime_config] schema` name thread into the DDL applicator at construction
-and carry across refreshes unchanged. `replicate_all` (default `true`)
+and carry across refreshes unchanged. Refreshed snapshots provide per-table
+destinations and scope. `replicate_all` (default `true`)
 auto-creates and replicates every user table whose namespace is not a system
 schema (`pg_*`, `information_schema`, the runtime-config schema); an explicit
 `auto_create` namespace or a `replicate = false` opt-out still wins.
@@ -138,6 +135,53 @@ whether the daemon owns a shadow lifecycle at all, which is a
 per-invocation recovery decision like `--start-lsn` and `--ignore-cursor`,
 and CLI-only keeps a stale config file from stomping it.
 
+## Name patterns
+
+Set `match` on a table or column entry to choose how names are read:
+
+- `exact` matches a literal name and is default
+- `glob` supports `*`, `?`, character classes such as `[a-z]`, and choices
+  such as `{one,two}`
+- `regex` supports regular expressions without backreferences
+
+Glob and regex patterns match whole names. For example, `events_*` matches
+`events_2026`, but `events` does not match `my_events`.
+
+```toml
+[table.app."events_*"]
+match = "glob"
+replicate = true
+initial_load = "copy"
+target_database = "warehouse"
+```
+
+```sql
+INSERT INTO walshadow.config_table (namespace, relname, match, replicate)
+VALUES ('app', 'events_*', 'glob', true);
+```
+
+Use glob for common prefix and suffix matches. Use regex when glob cannot
+express a pattern, for example `v[0-9]+_.*`.
+
+Matching patterns apply from least specific to most specific. Longer combined
+namespace and table patterns are more specific. An exact entry applies last.
+Runtime config wins when it repeats a TOML pattern. Any matching pattern with
+`replicate = false` blocks pattern-based opt-ins, but an exact entry can
+override it.
+
+`config_column.match` applies to namespace, table, and column names together.
+Invalid patterns and unknown match modes are rejected and logged. Other rules
+remain active.
+
+Pattern table rules also control scope:
+
+- `replicate = false` prevents automatic creation and removes matching routes
+- `replicate = true` enables automatic creation and requested initial loads
+  for matching tables, including tables created later
+- existing routes keep pinned column mappings
+- patterns never include `pg_*`, `information_schema`, or runtime config
+  schemas
+
 ## `<schema>.config_*` tables
 
 DBA runs [`sql/runtime_config_install.sql`](../sql/runtime_config_install.sql)
@@ -150,12 +194,14 @@ daemon — preserving walshadow's read-only-source posture. Four tables:
   `drop_table_strategy`
 - `config_namespace` — key `namespace`: `target_database`, `auto_create`,
   `drop_table_strategy`
-- `config_table` — key `(namespace, relname)`: `target_database`,
-  `target_table` (each NULL = derived: namespace default / source relname),
-  `replicate`, `initial_load` (`none`, `copy`, `base_backup`, `object_store`).
+- `config_table` — key `(namespace, relname)`: `match` (`exact`, `glob`, or
+  `regex`), `target_database`, `target_table`
+  (each NULL = derived: namespace default / source relname), `replicate`,
+  `initial_load` (`none`, `copy`, `base_backup`, `object_store`).
   Name key, not relfilenode, rfn is unknown at row-insert time for
   forward-declared tables
-- `config_column` — key `(namespace, relname, attname)`: `target_type`
+- `config_column` — key `(namespace, relname, attname)`: `match` (`exact`,
+  `glob`, or `regex`), `target_type`
 
 Every column is nullable and NULL means "daemon default / TOML applies", so the
 schema grows additively: a newer daemon reading an older install still works.
@@ -277,6 +323,16 @@ destination unchanged.
 
 ## Column overrides
 
+Column rules use same precedence as name patterns: least specific patterns
+first, followed by an exact rule. When TOML and `config_column` define same
+rule, `config_column` wins.
+
+- TOML name rules may set ClickHouse column names and types. New tables and
+  columns use these values in mappings and DDL. A custom type removes any
+  generated default. Nullable columns are omitted from `ORDER BY`.
+- `config_column` may change encoder type for an existing column. It cannot
+  rename a ClickHouse column or alter its type.
+
 `config_column.target_type` reaches the emitted projection in two stages,
 because the two failure classes surface at different points:
 
@@ -306,9 +362,8 @@ receiver, so backfilled rows encode under the same overrides as WAL-driven
 rows. The greenfield bootstrap tail stays TOML-only (no resolver exists yet at
 that phase).
 
-The override changes the projection only — CH-side DDL (`CREATE TABLE` /
-`ADD COLUMN`) still renders bridge-derived types; retyping an existing CH
-column stays an operator migration.
+Runtime overrides change encoder projection only. Retyping an existing
+ClickHouse column remains an operator migration.
 
 ## Subscribers
 

plans/emitter.md

@@ -436,6 +436,29 @@ columns = [
 ]
 ```
 
+A `columns` entry uses either `attnum` or `name`. Do not mix both forms in one
+array.
+
+- `attnum` pins a projection. `target` and `type` are required.
+- `name` changes a catalog-derived column. `target` and `type` are optional.
+  Source name and derived type remain when omitted. Set `match` to `glob` or
+  `regex` to cover current and future columns. Pattern entries cannot set
+  `target`, because several columns cannot share one ClickHouse name.
+
+```toml
+[table.app."*"]
+match = "glob"
+replicate = true
+columns = [
+  { name = "*_at", match = "glob", type = "DateTime64(6, 'UTC')" },
+  { name = "legacy_id", target = "id" },
+]
+```
+
+Name entries do not select tables for replication. Use `replicate` or
+`auto_create` for scope. Column rules apply to mappings, `CREATE TABLE`,
+`ADD COLUMN`, and encoder plans.
+
 `MappingHandle = Arc<tokio::sync::RwLock<HashMap<RelName, TableMapping>>>`
 is the live handle the planner's route view resolves from. Handle is
 cloneable; daemon's SIGHUP task swaps whole inner `HashMap`. Routes

sql/runtime_config_install.sql

@@ -42,6 +42,7 @@ CREATE TABLE IF NOT EXISTS :"walshadow_schema".config_namespace (
 CREATE TABLE IF NOT EXISTS :"walshadow_schema".config_table (
     namespace       text NOT NULL,
     relname         text NOT NULL,
+    match           text,
     target_database text,            -- ClickHouse database; NULL derives from
                                      -- config_namespace.target_database / TOML
     target_table    text,            -- ClickHouse table; NULL derives from relname
@@ -59,6 +60,7 @@ CREATE TABLE IF NOT EXISTS :"walshadow_schema".config_column (
     namespace   text NOT NULL,
     relname     text NOT NULL,
     attname     text NOT NULL,
+    match       text,
     target_type text,                -- ClickHouse type expression
     PRIMARY KEY (namespace, relname, attname)
 );
@@ -70,6 +72,8 @@ ALTER TABLE :"walshadow_schema".config_table ADD COLUMN IF NOT EXISTS replicate
 ALTER TABLE :"walshadow_schema".config_table ADD COLUMN IF NOT EXISTS initial_load    text;
 ALTER TABLE :"walshadow_schema".config_table ADD COLUMN IF NOT EXISTS target_database text;
 ALTER TABLE :"walshadow_schema".config_table ADD COLUMN IF NOT EXISTS target_table    text;
+ALTER TABLE :"walshadow_schema".config_table ADD COLUMN IF NOT EXISTS match           text;
+ALTER TABLE :"walshadow_schema".config_column ADD COLUMN IF NOT EXISTS match           text;
 
 -- REPLICA IDENTITY FULL logs the complete old-row image on UPDATE/DELETE, so a
 -- DELETE always carries the key columns the decoder reads (namespace/relname/

src/backfill/backup_backfill.rs

@@ -1080,18 +1080,12 @@ impl ReplaySink {
                         continue;
                     };
                     let mapping = Arc::new(mapping.clone());
-                    let overrides = self
+                    let rules = self
                         .config
                         .as_ref()
-                        .and_then(|rc| rc.columns.get(&rel.rel_name))
-                        .cloned()
-                        .map(Arc::new)
-                        .unwrap_or_default();
-                    let route = crate::emit::route::RouteSnapshot::freeze(
-                        mapping,
-                        overrides,
-                        self.soft_delete,
-                    );
+                        .map_or_else(Arc::default, |rc| rc.column_rules.clone());
+                    let route =
+                        crate::emit::route::RouteSnapshot::freeze(mapping, rules, self.soft_delete);
                     let seq = if let Some((seq, rows)) = &mut self.open {
                         *rows += 1;
                         *seq