Skip to content

Fail on columns the contract does not declare, when the schema says so - #1725

Open
markhm wants to merge 1 commit into
datacontract:mainfrom
markhm:feat/no-additional-fields
Open

markhm wants to merge 1 commit into
datacontract:mainfrom
markhm:feat/no-additional-fields

Conversation

@markhm

@markhm markhm commented Oct 10, 2026

Copy link
Copy Markdown

Fixes #1722.

Problem

A contract cannot say that a file must have no columns besides the ones it declares. An extra column passes unnoticed. ODCS v3.2.0 has no field for this.

What this changes

A schema opts in with a custom property, named after the same idea in JSON Schema:

schema:
  - name: orders
    customProperties:
      - property: additionalProperties
        value: false
  • The schema then gets a check, "Check that orders has no fields the contract does not declare" (type model_no_additional_fields, dimension conformity). It fails on columns the schema does not declare and names them: Fields not in the contract: note, with additional_fields in the diagnostics.
  • Names compare case-insensitively, like the existing field presence check.
  • It reads only the column names, never the rows, so it also runs with --metadata-only. On a CSV file it reads the {model}__raw__ view; elsewhere the table itself.
  • It works on every server read through ibis: CSV, JSON and Parquet files, and database tables. SAP HANA has its own engine and is not covered.
  • The value may be the boolean false or the text "false".
  • Without the custom property nothing changes.

Tests

tests/test_test_additional_fields.py: 9 tests, covering an extra column, exactly the declared columns, case, no custom property, text values, a JSON file, --metadata-only, and a DuckDB database table. uv run pytest: all pass. ruff: clean.

Docs

schema.md: a row in the checks table and a short section on the setting. CHANGELOG entry under Added.

Note

This PR and #1724 touch a comment in check_spec.py and the same bullet in schema.md, so the one merged second needs a small rebase. I'll do that.

  • Tests pass (uv run pytest)
  • Code formatted (uv run ruff check --fix && uv run ruff format)
  • Docs updated (if relevant)
  • CHANGELOG.md entry added

🤖 Generated with Claude Code

A schema with the custom property `additionalProperties: false` (the JSON
Schema term; ODCS has no field for it) gets a check that fails on fields of
the data the schema does not declare, and names them. It reads only the
column names, so it runs on every server read through ibis, also with
--metadata-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSV: no way to fail on columns that the contract does not declare

1 participant