Skip to content

Implement native path translation for JsonNode navigation#13

Merged
alex-clickhouse merged 2 commits into
ClickHouse:mainfrom
HappyEntity:feature/json-node-translation
May 12, 2026
Merged

Implement native path translation for JsonNode navigation#13
alex-clickhouse merged 2 commits into
ClickHouse:mainfrom
HappyEntity:feature/json-node-translation

Conversation

@HappyEntity
Copy link
Copy Markdown
Contributor

@HappyEntity HappyEntity commented Apr 26, 2026

Description:

This PR closes the standing limitation #1 mentioned in the README: "No JSON path translation — entity.Payload["name"] in LINQ does not translate to ClickHouse's data.name SQL syntax.".

Now, the provider supports native ClickHouse navigation for System.Text.Json.Nodes.JsonNode using standard C# indexing and casting.

Key Features:

  • Property Access: Data["key"] translates to data.key.
  • Array Indexing: Data[0] translates to data[1] (handles ClickHouse 1-based indexing).
  • Deep Nesting: Supports infinite nesting like Data["user"]["address"]["city"].
  • Type Casting: Supports both .GetValue<T>() and explicit casting (T)Data["key"].
  • Matrix Support: Handles multi-dimensional arrays (e.g., Data["matrix"][0][0]).

Technical Changes:

  • Implemented ClickHouseJsonPathExpression and ClickHouseJsonArrayIndexExpression to represent JSON paths in the SQL tree.
  • Added ClickHouseJsonNodeTranslator with strict MethodInfo mapping to handle indexing and value extraction.
  • Updated ClickHouseQuerySqlGenerator with dedicated visitors for JSON paths and indices. Implemented custom VisitSqlUnary logic to handle explicit/implicit casting, ensuring clean CAST(... AS Type) output and avoiding ClickHouse "Illegal type JSON" errors.
  • Updated ClickHouseSqlNullabilityProcessor to ensure correct null propagation for JSON paths.

Examples:

LINQ:

.Where(e => (int)e.Data["age"] > 25)
.Select(e => new {
    City = e.Data["address"]["city"].GetValue<string>(),
    FirstTag = e.Data["tags"][0].GetValue<string>()
})

Generated SQL:

SELECT CAST(j.data.address.city AS String), CAST(j.data.tags[1] AS String)
FROM json_table AS j
WHERE CAST(j.data.age AS Int32) > 25

Note

On Object Projections:

Due to ClickHouse's native JSON type implementation (which uses dynamic sub-columns and flattens nested objects), projecting an entire intermediate object like Data["address"] will return an empty result or a flattened map. This is consistent with native ClickHouse behavior: SELECT data.address returns an empty string/map because the engine only physically stores leaf nodes like data.address.city.

However, projecting array elements containing objects (e.g., Data["orders"][0]) works as expected because ClickHouse preserves the internal structure for array elements.

Recommendation: To ensure predictable results, always project to the leaf property (e.g., (T)Data["address"]["city"]) or use .GetValue<T>().

Testing:

Added comprehensive integration tests in JsonNodeTranslationTests.cs covering:

  • Scalar types (string, int, long, double, bool, ulong).
  • Missing keys (verified ClickHouse implicit defaults behavior).
  • Deeply nested objects and matrix arrays.
  • Full object/array projections.

…cess

- Implemented ClickHouseJsonPathExpression and ClickHouseJsonArrayIndexExpression for SQL tree.
- Added ClickHouseJsonNodeTranslator to handle JsonNode["key"], JsonNode[index], and GetValue<T>().
- Updated ClickHouseQuerySqlGenerator to support native dot-notation and 1-based array indexing.
- Enhanced ClickHouseSqlNullabilityProcessor to handle JSON path nullability.
- Enabled support for explicit casting (e.g., (int)node["key"]) by implementing custom VisitSqlUnary logic to ensure clean SQL output and correct type mapping.
- Added comprehensive integration tests covering deep nesting, matrix access, and implicit defaults.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

@alex-clickhouse
Copy link
Copy Markdown
Collaborator

Looks good to me, please add a mention of the new features to CHANGELOG.md and RELEASENOTES.md and it's ready to go.

@HappyEntity
Copy link
Copy Markdown
Contributor Author

I've pushed the requested changes:

  • Updated CHANGELOG.md and RELEASENOTES.md under the new v0.3.0 section.

@alex-clickhouse
Copy link
Copy Markdown
Collaborator

Thank you for your contribution!

@alex-clickhouse alex-clickhouse merged commit d098c5f into ClickHouse:main May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants