Global caches cause unbounded heap growth in long-running processes

[sohamsengupta17]

Summary

libopenapi's process-wide caches are keyed by *yaml.Node and model-object pointer values. This creates two compounding problems for long-running processes:

1. Every new Document grows the caches permanently. Even when parsing the same spec repeatedly, each parse produces new heap allocations at new addresses, so every node is a fresh cache miss. Old entries are never evicted, and because the cache holds the pointer as a key, it keeps the entire prior YAML parse tree and model graph alive — preventing GC even after the caller drops all references.

2. Callers are forced to cache Document objects themselves. The only way to keep memory stable when validating specs seems to be reusing single long-lived Document or Validator instances. There is no way to create a short-lived Document per request or to disable the global caching behaviour altogether.

3. No fine-grained cache eviction. The only current workaround is the libopenapi.ClearAllCaches() which is a blunt instrument that offers no handle on specific documents or specs. It needs to be called often in long-running servers, hence requiring race condition handling with the validation flow and affecting runtime performance due to synchronization and cache rebuild overheads.

Background

Several caches in libopenapi seem to be process-global sync.Map or map instances, keyed by the pointer values of *yaml.Node objects (like nodeHashCache/hashCache) or high-level model objects (like indexCollectionCache, contentDetectionCache, etc).

Because nodeHashCache and hashCache hold *yaml.Node and model-object pointers as both key and implicit anchor, they prevent the GC from collecting any *yaml.Node or model object that has ever been hashed — even after the caller has dropped all other references to the Document.

The caches are well-suited only to a single long-lived Document, but cause unchecked accumulation in map entries for long-lived servers handling multiple validation calls for different api specs. This constraint is also invisible to callers and is not enforced or documented at the API level — it is easy to write code that creates a fresh Document per request and a clean slate on GC clear, causing monotonic heap growth and finally an out-of-memory crash.

Reproduction scenarios

Scenario 1: same spec, new Document per request

The natural way to write a request handler — parse the spec once per request, or on each config reload — causes monotonic heap growth even though it is always the same spec bytes:

// Looks reasonable. Each request gets a fresh, independent Document.
// In practice: heap grows on every call and never falls.
func handleRequest(specBytes []byte, req *http.Request) {
    doc, _ := libopenapi.NewDocument(specBytes)
    model, _ := doc.BuildV3Model()
    validator, _ := validator.NewValidatorFromV3Model(&model.Model)
    validator.ValidateHttpRequest(req)
    // doc goes out of scope — but nodeHashCache, hashCache,
    // indexCollectionCache, SchemaQuickHashMap, and inlineRenderingTracker
    // still hold every *yaml.Node and model object from this parse,
    // keeping the entire YAML tree and model graph alive indefinitely.
}

Scenario 2: multiple distinct specs (multi-tenant / API gateway)

When each tenant or upstream service has its own spec, the leak compounds: each new spec adds a full document's worth of nodes to the global caches, permanently:

for _, specBytes := range tenantSpecs {
    doc, _ := libopenapi.NewDocument(specBytes)
    model, _ := doc.BuildV3Model()
    validate(model)
    // Memory from this doc is never reclaimed. nodeHashCache,
    // hashCache, indexCollectionCache, SchemaQuickHashMap,
    // contentDetectionCache, and inlineRenderingTracker all
    // grow with each iteration.
}

Calling libopenapi.ClearAllCaches() between iterations works, but requires the caller to know about internal implementation details, is a global reset that cannot be safely called while any other goroutine is mid-parse, cannot be targeted at a single document, and makes the global caching somewhat pointless.

Proposed solutions

Three solution approaches proposed below:

1. Per-document opt-out via DocumentConfiguration

Add a flag to DocumentConfiguration to disable hash caching entirely for a document. Callers who don't use CompareDocuments pay no caching overhead at all. This option should have zero breaking changes.

type DocumentConfiguration struct {
    // ...existing fields...
    DisableHashCaching bool // skip nodeHashCache/hashCache/SchemaQuickHashMap population
}

2. Injectable cache interfaces

Expose the caches behind interfaces (similar to how libopenapi-validator already does for SchemaCache and RegexCache) so callers can supply an LRU-bounded or no-op implementation. The same pattern would apply to all affected caches:

type NodeHashCache interface {
    Load(key *yaml.Node) (string, bool)
    Store(key *yaml.Node, value string)
}

type ModelHashCache interface {
    Load(key uintptr) (string, bool)
    Store(key uintptr, value string)
}

type DocumentConfiguration struct {
    // ...
    NodeHashCache  NodeHashCache  // nil = no caching
    ModelHashCache ModelHashCache // nil = no caching
    // SchemaQuickHashMap, indexCollectionCache, inlineRenderingTracker follow the same pattern
}

This also makes the caches testable and allows callers to implement size-bounded eviction policies appropriate to their workload.

3. Scope caches to the document or index

Move the caches off the package level and onto SpecIndex or DocumentModel. Each document owns its own cache, which is naturally collected when the document is released. In this solution there would be no global state, no coordination required and GC is utilized.

type SpecIndex struct {
    // ...
    nodeHashCache sync.Map // was package-level
}

This should still retain the benefit of fast access to the yaml nodes for BuildV3Model() and validation during the lifetime of the document. For short-lived documents, there would be some re-hashing cost per document construction, but we would get bounded memory which is more critical for application stability.

Prior art in libopenapi-validator

libopenapi-validator solves the equivalent problem for JSON schema compilation in pb33f/libopenapi-validator#187. Before that change, the validator was allocating ~100KB of schema objects per request; after it, request validation is 6.5× faster with 90% less memory.

The design it settled on is exactly what is proposed here for libopenapi:

• A SchemaCache interface with Load/Store/Release methods, backed by a sync.Map default
• A SchemaResourceCache interface for rendered document-level resources, same pattern
• Exposed via ValidationOptions.SchemaCache / ValidationOptions.SchemaResourceCache
• Disable by passing nil — all cache call sites nil-check safely, so callers that don't need caching pay nothing
• Replace with a custom implementation — callers can supply an LRU-bounded cache, a no-op, or anything else that satisfies the interface

libopenapi does not have the same memory problem on the validator side is because of that PR. The identical fix is needed here at the core parsing layer.

Current workaround

// Safe only when no concurrent parses are running.
defer libopenapi.ClearAllCaches()

This is documented in cache.go but requires callers to understand libopenapi internals and is unsuitable for concurrent or multi-tenant use. It also does not offer any fine-grained per-document handles as noted above. For multi-tenant scenarios with different API specs to be validated, this also affects performance due to the cache rebuild and clear overhead.

Impact

Any service that creates a Document more than once — whether processing requests against the same spec or validating across multiple specs — is affected. The caches grow proportionally to the cumulative number of *yaml.Node allocations across all parses, with no upper bound and no GC relief - ultimately leading to OOMKill errors. Keeping a singleton Document or Validator cached for the process lifetime, and calling ClearAllCaches() periodically is not suitable for multiple reasons as noted above, especially in multi-tenant, multi-spec scenarios.