You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, we are using your library (which is wonderful by the way) and I run into a race condition with -race on BuildV3Model in my tests
func LoadOpenAPIModel(openAPIRoot string, openAPIPath string, opts *DocOptions) (*libopenapi.DocumentModel[v3.Document], map[string]*yaml.Node, error) {
data, errRead := os.ReadFile(openAPIPath)
if errRead != nil {
return nil, nil, fmt.Errorf("cannot read file: %e", errRead)
}
dir := filepath.Dir(openAPIPath)
// the CWD when doc is created is the scope in which the refs can be located
// so if refs are located in the parent directory of the spec, we need to change the CWD
// to the parent directory.
if errChdir := os.Chdir(openAPIRoot); errChdir != nil {
return nil, nil, fmt.Errorf("cannot change directory to %s: %e", openAPIRoot, errChdir)
}
config := datamodel.DocumentConfiguration{
AllowFileReferences: true,
BasePath: dir,
// Avoids a data race condition
ExtractRefsSequentially: true,
}
if len(opts.IncludeTags) > 0 {
data = onlyIncludeTags(data, opts.IncludeTags)
}
// We can't use github.com/pb33f/libopenapi/bundler#BundleBytes here to simply bundle
// everything into a single file for ease of transform because it also
// inlines most schemas, but the openapi generator requires schemas like enums
// to be separate named schema objects, otherwise they get interpreted as
// simple strings.
document, errNewDoc := libopenapi.NewDocumentWithConfiguration(data, &config)
if errNewDoc != nil {
return nil, nil, fmt.Errorf("cannot create new document: %e", errNewDoc)
}
//document.GetRolodex().AddLocalFS(dir, fs.NewGlobalFS(dir))
docModel, errV3 := document.BuildV3Model()
if len(errV3) > 0 {
for i := range errV3 {
fmt.Printf("error: %e\n", errV3[i])
}
return nil, nil, fmt.Errorf("cannot create v3 model from document: %d errors reported", len(errV3))
}
refFiles := map[string]*yaml.Node{}
// gets files linked in $refs, includes indirect references
for _, index := range docModel.Index.GetRolodex().GetIndexes() {
refFileBytes, _ := os.ReadFile(index.GetSpecAbsolutePath())
var refFileNode yaml.Node
if errUnmarshal := yaml.Unmarshal(refFileBytes, &refFileNode); errUnmarshal != nil {
return nil, nil, fmt.Errorf("cannot unmarshal ref file: %e", errUnmarshal)
}
refFiles[index.GetSpecAbsolutePath()] = &refFileNode
}
return docModel, refFiles, nil
}
Based on what I can tell, with
ExtractRefsSequentially: false
A go thread is spawned for each ref. I assume that's because for remote refs and stuff like the comment inside mentions, it can take a bit to resolve.
My assumption since it only seems to occur with sequential tests is that there are lingering go threads spawned by the ref resolver, and these persist between function runs. I wanted to make a PR to possibly fix it, but I wasn't sure what the intended behavior was or if we were using it wrong. I was thinking two possible solutions would be to expose a channel to wait on, or make a sister function that took in a context that we could cancel between tests or wait on.
The text was updated successfully, but these errors were encountered:
Hi, we are using your library (which is wonderful by the way) and I run into a race condition with -race on BuildV3Model in my tests
Based on what I can tell, with
A go thread is spawned for each ref. I assume that's because for remote refs and stuff like the comment inside mentions, it can take a bit to resolve.
My assumption since it only seems to occur with sequential tests is that there are lingering go threads spawned by the ref resolver, and these persist between function runs. I wanted to make a PR to possibly fix it, but I wasn't sure what the intended behavior was or if we were using it wrong. I was thinking two possible solutions would be to expose a channel to wait on, or make a sister function that took in a context that we could cancel between tests or wait on.
The text was updated successfully, but these errors were encountered: