Skip to content

Commit

Permalink
Intersection schema generation is order-dependent (#603)
Browse files Browse the repository at this point in the history
* Add new test cases.

* Intersection schema generation is order-dependent

- Given a schema that contains a named definition (`B`),
- And that named definition is referenced in multiple locations,
- And that named schema is also an intersection type (`allOf` in this
  example),

Then when parsed, the generated TypeScript will contain the correct
reference only for the _first_ location in which the named schema is
encountered, during a depth-first traversal.

Subsequent references to the same schema will be generated as though
they were only the intersection type, and not the named schema.

Example

Given the following schema:

```yaml
$id: Intersection
type: object
oneOf:
  - $ref: '#/definitions/A'
  - $ref: '#/definitions/B'

definitions:
  A:
    type: object
    additionalProperties: false
    allOf: [$ref: '#/definitions/Base']
    properties:
      b: {$ref: '#/definitions/B'}
  B:
    type: object
    additionalProperties: false,
    allOf: [$ref: '#/definitions/Base']
    properties:
      x: {type: string}
  Base:
    type: object
    additionalProperties: false,
    properties:
      y: {type: string}
```

The current resulting TypeScript will be (comments adjusted
for clarity):

```ts
// Incorrect: should be `type Intersection = A | B`
// Note that the B type at this location is the _second_ reference to
// B during a depth-first traversal.
export type Intersection = A | B1;
export type A = A1 & {
  b?: B;
};
export type A1 = Base;
export type B = B1 & {
  x?: string;
};
export type B1 = Base;

export interface Base {
  y?: string;
}
```

Root Cause

In `parser.ts`, [lines 57 - 75][1], when schema that matches multiple
"types" is encountered, the parser generates a new `ALL_OF` intersection
schema to contain each sub-type, then adds each sub-type to the new
`ALL_OF` schema.

Each sub-type is then parsed sequentially. During this process,
`maybeStripNameHints` is called, which mutates the schema by removing
the `$id`, `description`, and `name` properties.

Notably, these properties are used by `typesOfSchema` to detect the
`NAMED_SCHEMA` type. As a result, this schema object will never again be
detected as a `NAMED_SCHEMA` type.

Therefore, the _first_ instance of the schema object is correctly
handled as an intersection schema **and** a named schema, but all
subsequent instances are treated as though they are **only** an
intersection schema.

Proposed Solution

- The call to `typesOfSchema` is moved from `parser.ts` to
  `normalizer.ts`, with the goal of avoiding confusion due to a mutated
  schema object. The resulting list of schema types is persisted on the
  schema using a newly-introduced `Types` symbol.

- The generated intersection schema is _also_ moved from `parser.ts` to
  `normalizer.ts`. This is because it is advantageous to let the
  generated intersection schema participate in the caching mechanism
  (which it could not previously do, since it was generated dynamically
  during each encounter). Without this, multiple instances of the same
  schema are generated.

Related Issues

- #597

[1]: https://github.com/bcherny/json-schema-to-typescript/blob/31993def993b610ba238d3024260129e31ddc371/src/parser.ts#L57-L75 'parser.ts, lines 57 - 75'

* Additionally hoist `allOf` behavior.

* Traverse the generated intersection schema.
  • Loading branch information
altearius authored Jul 22, 2024
1 parent e0a822f commit 62cc052
Show file tree
Hide file tree
Showing 12 changed files with 977 additions and 790 deletions.
51 changes: 51 additions & 0 deletions src/applySchemaTyping.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import type {LinkedJSONSchema} from './types/JSONSchema'
import {Intersection, Parent, Types} from './types/JSONSchema'
import {typesOfSchema} from './typesOfSchema'

export function applySchemaTyping(schema: LinkedJSONSchema) {
const types = typesOfSchema(schema)

Object.defineProperty(schema, Types, {
enumerable: false,
value: types,
writable: false,
})

if (types.size === 1) {
return
}

// Some schemas can be understood as multiple possible types (see related
// comment in `typesOfSchema.ts`). In such cases, we generate an `ALL_OF`
// intersection that will ultimately be used to generate a union type.
//
// The original schema's name, title, and description are hoisted to the
// new intersection schema to prevent duplication.
//
// If the original schema also contained its own `ALL_OF` property, it is
// also hoiested to the new intersection schema.
const intersection = {
[Parent]: schema,
[Types]: new Set(['ALL_OF']),
$id: schema.$id,
description: schema.description,
name: schema.name,
title: schema.title,
allOf: schema.allOf ?? [],
required: [],
additionalProperties: false,
}

types.delete('ALL_OF')
delete schema.allOf
delete schema.$id
delete schema.description
delete schema.name
delete schema.title

Object.defineProperty(schema, Intersection, {
enumerable: false,
value: intersection,
writable: false,
})
}
17 changes: 17 additions & 0 deletions src/normalizer.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import {JSONSchemaTypeName, LinkedJSONSchema, NormalizedJSONSchema, Parent} from './types/JSONSchema'
import {appendToDescription, escapeBlockComment, isSchemaLike, justName, toSafeString, traverse} from './utils'
import {Options} from './'
import {applySchemaTyping} from './applySchemaTyping'
import {DereferencedPaths} from './resolver'
import {isDeepStrictEqual} from 'util'

Expand Down Expand Up @@ -231,6 +232,22 @@ rules.set('Add tsEnumNames to enum types', (schema, _, options) => {
}
})

// Precalculation of the schema types is necessary because the ALL_OF type
// is implemented in a way that mutates the schema object. Detection of the
// NAMED_SCHEMA type relies on the presence of the $id property, which is
// hoisted to a parent schema object during the ALL_OF type implementation,
// and becomes unavailable if the same schema is used in multiple places.
//
// Precalculation of the `ALL_OF` intersection schema is necessary because
// the intersection schema needs to participate in the schema cache during
// the parsing step, so it cannot be re-calculated every time the schema
// is encountered.
rules.set('Pre-calculate schema types and intersections', schema => {
if (schema !== null && typeof schema === 'object') {
applySchemaTyping(schema)
}
})

export function normalize(
rootSchema: LinkedJSONSchema,
dereferencedPaths: DereferencedPaths,
Expand Down
87 changes: 30 additions & 57 deletions src/parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,19 @@ import {JSONSchema4Type, JSONSchema4TypeName} from 'json-schema'
import {findKey, includes, isPlainObject, map, memoize, omit} from 'lodash'
import {format} from 'util'
import {Options} from './'
import {typesOfSchema} from './typesOfSchema'
import {
AST,
T_ANY,
T_ANY_ADDITIONAL_PROPERTIES,
TInterface,
TInterfaceParam,
TNamedInterface,
TTuple,
T_UNKNOWN,
T_UNKNOWN_ADDITIONAL_PROPERTIES,
TIntersection,
} from './types/AST'
import {
import {applySchemaTyping} from './applySchemaTyping'
import type {AST, TInterface, TInterfaceParam, TIntersection, TNamedInterface, TTuple} from './types/AST'
import {T_ANY, T_ANY_ADDITIONAL_PROPERTIES, T_UNKNOWN, T_UNKNOWN_ADDITIONAL_PROPERTIES} from './types/AST'
import type {
EnumJSONSchema,
getRootSchema,
isBoolean,
isPrimitive,
JSONSchemaWithDefinitions,
LinkedJSONSchema,
NormalizedJSONSchema,
Parent,
SchemaSchema,
SchemaType,
} from './types/JSONSchema'
import {generateName, log, maybeStripDefault, maybeStripNameHints} from './utils'
import {Intersection, Types, getRootSchema, isBoolean, isPrimitive} from './types/JSONSchema'
import {generateName, log, maybeStripDefault} from './utils'

export type Processed = Map<NormalizedJSONSchema, Map<SchemaType, AST>>

Expand All @@ -47,40 +35,28 @@ export function parse(
return parseLiteral(schema, keyName)
}

const types = typesOfSchema(schema)
if (types.length === 1) {
const ast = parseAsTypeWithCache(schema, types[0], options, keyName, processed, usedNames)
log('blue', 'parser', 'Types:', types, 'Input:', schema, 'Output:', ast)
const intersection = schema[Intersection]
const types = schema[Types]

if (intersection) {
const ast = parseAsTypeWithCache(intersection, 'ALL_OF', options, keyName, processed, usedNames) as TIntersection

types.forEach(type => {
ast.params.push(parseAsTypeWithCache(schema, type, options, keyName, processed, usedNames))
})

log('blue', 'parser', 'Types:', [...types], 'Input:', schema, 'Output:', ast)
return ast
}

// Be careful to first process the intersection before processing its params,
// so that it gets first pick for standalone name.
const ast = parseAsTypeWithCache(
{
[Parent]: schema[Parent],
$id: schema.$id,
additionalProperties: schema.additionalProperties,
allOf: [],
description: schema.description,
required: schema.required,
title: schema.title,
},
'ALL_OF',
options,
keyName,
processed,
usedNames,
) as TIntersection

ast.params = types.map(type =>
// We hoist description (for comment) and id/title (for standaloneName)
// to the parent intersection type, so we remove it from the children.
parseAsTypeWithCache(maybeStripNameHints(schema), type, options, keyName, processed, usedNames),
)

log('blue', 'parser', 'Types:', types, 'Input:', schema, 'Output:', ast)
return ast
if (types.size === 1) {
const type = [...types][0]
const ast = parseAsTypeWithCache(schema, type, options, keyName, processed, usedNames)
log('blue', 'parser', 'Type:', type, 'Input:', schema, 'Output:', ast)
return ast
}

throw new ReferenceError('Expected intersection schema. Please file an issue on GitHub.')
}

function parseAsTypeWithCache(
Expand Down Expand Up @@ -293,13 +269,10 @@ function parseNonLiteral(
keyName,
standaloneName: standaloneName(schema, keyNameFromDefinition, usedNames, options),
params: (schema.type as JSONSchema4TypeName[]).map(type => {
const member: NormalizedJSONSchema = {
...omit(schema, '$id', 'description', 'title'),
type,
additionalProperties: schema.additionalProperties,
required: schema.required,
}
return parse(maybeStripDefault(member as any), options, undefined, processed, usedNames)
const member: LinkedJSONSchema = {...omit(schema, '$id', 'description', 'title'), type}
maybeStripDefault(member)
applySchemaTyping(member)
return parse(member, options, undefined, processed, usedNames)
}),
type: 'UNION',
}
Expand Down
5 changes: 5 additions & 0 deletions src/types/JSONSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,18 @@ export interface LinkedJSONSchema extends JSONSchema {
not?: LinkedJSONSchema
}

export const Types = Symbol('Types')
export const Intersection = Symbol('Intersection')

/**
* Normalized JSON schema.
*
* Note: `definitions` and `id` are removed by the normalizer. Use `$defs` and `$id` instead.
*/
export interface NormalizedJSONSchema extends Omit<LinkedJSONSchema, 'definitions' | 'id'> {
[Intersection]?: NormalizedJSONSchema
[Parent]: NormalizedJSONSchema | null
[Types]: ReadonlySet<SchemaType>

additionalItems?: boolean | NormalizedJSONSchema
additionalProperties: boolean | NormalizedJSONSchema
Expand Down
14 changes: 7 additions & 7 deletions src/typesOfSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,26 @@ import {isCompound, JSONSchema, SchemaType} from './types/JSONSchema'
* types). The spec leaves it up to implementations to decide what to do with this
* loosely-defined behavior.
*/
export function typesOfSchema(schema: JSONSchema): readonly [SchemaType, ...SchemaType[]] {
export function typesOfSchema(schema: JSONSchema): Set<SchemaType> {
// tsType is an escape hatch that supercedes all other directives
if (schema.tsType) {
return ['CUSTOM_TYPE']
return new Set(['CUSTOM_TYPE'])
}

// Collect matched types
const matchedTypes: SchemaType[] = []
const matchedTypes = new Set<SchemaType>()
for (const [schemaType, f] of Object.entries(matchers)) {
if (f(schema)) {
matchedTypes.push(schemaType as SchemaType)
matchedTypes.add(schemaType as SchemaType)
}
}

// Default to an unnamed schema
if (!matchedTypes.length) {
return ['UNNAMED_SCHEMA']
if (!matchedTypes.size) {
matchedTypes.add('UNNAMED_SCHEMA')
}

return matchedTypes as [SchemaType, ...SchemaType[]]
return matchedTypes
}

const matchers: Record<SchemaType, (schema: JSONSchema) => boolean> = {
Expand Down
43 changes: 22 additions & 21 deletions src/utils.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import {deburr, isPlainObject, trim, upperFirst} from 'lodash'
import {basename, dirname, extname, normalize, sep, posix} from 'path'
import {JSONSchema, LinkedJSONSchema, NormalizedJSONSchema, Parent} from './types/JSONSchema'
import {Intersection, JSONSchema, LinkedJSONSchema, NormalizedJSONSchema, Parent} from './types/JSONSchema'
import {JSONSchema4} from 'json-schema'
import yaml from 'js-yaml'

Expand Down Expand Up @@ -71,6 +71,26 @@ function traverseArray(
arr.forEach((s, k) => traverse(s, callback, processed, k.toString()))
}

function traverseIntersection(
schema: LinkedJSONSchema,
callback: (schema: LinkedJSONSchema, key: string | null) => void,
processed: Set<LinkedJSONSchema>,
) {
if (typeof schema !== 'object' || !schema) {
return
}

const r = schema as unknown as Record<string | symbol, unknown>
const intersection = r[Intersection] as NormalizedJSONSchema | undefined
if (!intersection) {
return
}

if (Array.isArray(intersection.allOf)) {
traverseArray(intersection.allOf, callback, processed)
}
}

export function traverse(
schema: LinkedJSONSchema,
callback: (schema: LinkedJSONSchema, key: string | null) => void,
Expand Down Expand Up @@ -130,6 +150,7 @@ export function traverse(
if (schema.not) {
traverse(schema.not, callback, processed)
}
traverseIntersection(schema, callback, processed)

// technically you can put definitions on any key
Object.keys(schema)
Expand Down Expand Up @@ -331,26 +352,6 @@ export function maybeStripDefault(schema: LinkedJSONSchema): LinkedJSONSchema {
return schema
}

/**
* Removes the schema's `$id`, `name`, and `description` properties
* if they exist.
* Useful when parsing intersections.
*
* Mutates `schema`.
*/
export function maybeStripNameHints(schema: NormalizedJSONSchema): NormalizedJSONSchema {
if ('$id' in schema) {
delete schema.$id
}
if ('description' in schema) {
delete schema.description
}
if ('name' in schema) {
delete schema.name
}
return schema
}

export function appendToDescription(existingDescription: string | undefined, ...values: string[]): string {
if (existingDescription) {
return `${existingDescription}\n\n${values.join('\n')}`
Expand Down
Loading

0 comments on commit 62cc052

Please sign in to comment.