Add support for different bundling modes - compose bundle mode

[stcalica]

This PR improves the OpenAPI bundler to correctly handle complex reference scenarios by resolving references without placing them inline and instead referencing them in the components and schema. The bundler now successfully creates a complete bundled document from specifications with nested and cross-component references.

Changes

• Implemented depth-first traversal for references to ensure dependencies are processed before dependents
• Added component registration in the index to make newly added components discoverable for reference resolution
• Optimized recursion by adding proper reference tracking to avoid redundant processing
• Fixed reference resolution for nested components (schemas referenced by responses, etc.)
• Added support for all component types (schemas, responses, requestBodies, callbacks, links, headers, etc.)

Technical Details

This is a continuation of the work done here form @felixjung

The current bundling approach in libopenapi automatically inlines all refs in files other than the root file. The BundleInlineRefs option in datamodel.DocumentConfiguration appears to only affect the root file. When set to true, refs in the root file are inlined. When set to false, refs within the root file, i.e. to the components section, are kept where they are, while refs made to other files are inlined nevertheless.

The goal of this PR is to introduce different bundling modes.

The first mode inlines all references.
The second mode tracks all references across all files and ensures that they are moved to the appropriate sub-section of the components section or other sections (as necessary) in the bundled spec.
The bundle mode can be configured in a new BundlerOptions struct passed to the bundling functions of the bundler package.

• Changed from the previous PR with a two-phase approach (collect references as an array then build components) to a single recursive pass that builds components in dependency order
• Added a registerComponentInIndex() function to update the index's mapped references after adding components to the model
• Implemented proper reference visitation tracking to avoid processing the same component multiple times
• Ensured all component types from the OpenAPI specification are supported (schemas, responses, parameters, examples, requestBodies, headers, securitySchemes, links, and callbacks)

Testing

Verified the bundler can now successfully bundle specifications with complex reference dependencies, including the test case with AccountResponse referencing Account.

Added a bundledRef test to test more advanced exploded specs

Next Steps

• Adding more functionality to make sure properties and fields are properly copied over
• Further optimization of reference resolution for large specifications
• Enhanced error reporting for issues with reference resolution
• Additional test cases for complex reference scenarios

[stcalica]

@daveshanley I realized while testing that some of the nodes don't carry over the description property in most of my nodes (I had to change my testing to checking for key name instead) .And other nodes don't carry over any or much other information at all.

Example here is:

    AccountCallback:
      newMessage:
        '{$request.body#/callbackUrl}':
          post:
            requestBody:
              description: Callback payload
              content:
                application/json:
                  schema:
                    type: object
                    properties:
                      message:
                        type: string
            responses:
              '200':
                description: Callback received

becomes this after bundling:

    callbacks:
        AccountCallback:
            newMessage: {}

I think this is cause I'm building the nodes in the low level and converting to high level, and either I'm not carrying over the proper fields or I'm missing a step somewhere.

For example, this is how I build the nodes and add them to the high level:

	case "callbacks":
		var callback lowV3.Callback
		ctx := context.Background()
		err := callback.Build(ctx, lowModel.Components.ValueNode, mappedRef.Node, idx) //build low level node
		if err != nil {
			return fmt.Errorf("failed to build response for component %s: %w", mappedRef.Definition, err)
		}
		highLevelCallback := highV3.NewCallback(&callback) //convert into high level node
		model.Components.Callbacks.Set(defParts[3], highLevelCallback) // add to high level model that is mutable 
		registerComponentInIndex(defParts[2], defParts[3], mappedRef.Node, idx) // register the node in index spec for resolver to find

Is there an example in the codebase I should be following? How do I properly build these nodes?

[califlower]

extremely interested in this PR, as it would finally solve recursive references being ignored when bundling. Bundling is my only complaint with libopenapi, as it's very simple.

Ideal scenario, we have the same option as redocly, and have a dereference option that can be true or false.

If true, we inline everything, but recursive refs go in components/schemas in the root doc. otherwise, everything just goes under components/schemas.

If you need any help in bringing this to 100%. I'd love to help out so it can get merged

[daveshanley]

@daveshanley I realized while testing that some of the nodes don't carry over the description property in most of my nodes (I had to change my testing to checking for key name instead) .And other nodes don't carry over any or much other information at all.

Example here is:

    AccountCallback:
      newMessage:
        '{$request.body#/callbackUrl}':
          post:
            requestBody:
              description: Callback payload
              content:
                application/json:
                  schema:
                    type: object
                    properties:
                      message:
                        type: string
            responses:
              '200':
                description: Callback received

becomes this after bundling:

    callbacks:
        AccountCallback:
            newMessage: {}

I think this is cause I'm building the nodes in the low level and converting to high level, and either I'm not carrying over the proper fields or I'm missing a step somewhere.

For example, this is how I build the nodes and add them to the high level:

	case "callbacks":
		var callback lowV3.Callback
		ctx := context.Background()
		err := callback.Build(ctx, lowModel.Components.ValueNode, mappedRef.Node, idx) //build low level node
		if err != nil {
			return fmt.Errorf("failed to build response for component %s: %w", mappedRef.Definition, err)
		}
		highLevelCallback := highV3.NewCallback(&callback) //convert into high level node
		model.Components.Callbacks.Set(defParts[3], highLevelCallback) // add to high level model that is mutable 
		registerComponentInIndex(defParts[2], defParts[3], mappedRef.Node, idx) // register the node in index spec for resolver to find

Is there an example in the codebase I should be following? How do I properly build these nodes?

I have to admit, this looks right to me. But then again, it's been a good long while since I looked at this part of the codebase.

The best place to look for examples on how to build high level objects from scratch is here: https://github.com/pb33f/libopenapi/tree/main/datamodel/high/v3

Look at the tests.

[stcalica]

@daveshanley thanks, digging in it might be that not all the nodes get created when doing this method, seems like another thing I can tackle as a separate task. but I'll confirm before I push. appreciate you taking a look <3

@califlower yeah, feel free to give this a pull and a test. I think I might be done with this soon. Would love more tests and verification but I'll reach out if I'm stuck again!

[stcalica]

@califlower I've pretty much burnt my engines out and still need to fix the merging and stuff.

Do you mind checking the tests in bundler/bundler.go?
Specifically TestBundleDocument_Circular

I'm failing it with:

                Error Trace:    /Users/kyle/Code/libopenapi/bundler/bundler_test.go:108
                Error:          "[{"time":"2025-05-21T17:36:08.275715-07:00","level":"WARN","msg":"[bundler] skipping circular reference","ref":"#/components/schemas/Two"} {"time":"2025-05-21T17:36:08.275792-07:00","level":"WARN","msg":"[bundler] skipping circular reference","ref":"#/components/schemas/Seven"} {"time":"2025-05-21T17:36:08.275794-07:00","level":"WARN","msg":"[bundler] skipping circular reference","ref":"#/components/schemas/Seven"} {"time":"2025-05-21T17:36:08.275797-07:00","level":"WARN","msg":"[bundler] skipping circular reference","ref":"#/components/schemas/Ten"} ]" should have 0 item(s), but has 5
                Test:           TestBundleDocument_Circular
--- FAIL: TestBundleDocument_Circular (0.00s)
FAIL
FAIL    github.com/pb33f/libopenapi/bundler     0.260s```

I know I am now logging circular reference skipping, but Dave wrote that in awhile back so no idea why I'm failing it now when he wasn't. 

Let me know if you have any ideas, will tackle this again later this week. 

[daveshanley]

This feature is not an easy one to build. It's why I have not dug in yet - I need a clear few days to really open up the brain on it.

[stcalica]

@daveshanley no problem. I actually have it working pretty well using BuildModel, Build, New*.
The complicated part is the byte counts tests and side effects, I'm churning through it all though now.

Let me know if you rather have me to throw this back to draft.

[califlower]

I'll try and take a look at this this weekend

[daveshanley]

This feature was implemented in v0.22

https://pb33f.io/libopenapi/bundling/

[califlower]

amazing!!

[stcalica]

@daveshanley oh wow, at least I learned something! Pretty new at Go as well so great to see how you tackled it. Thanks!