Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Nested arrays keys #84

Open
manifestori opened this issue Nov 26, 2023 · 3 comments
Open

[Bug]: Nested arrays keys #84

manifestori opened this issue Nov 26, 2023 · 3 comments
Labels
bug Something isn't working wontfix This will not be worked on

Comments

@manifestori
Copy link

manifestori commented Nov 26, 2023 •
edited
Loading

Description

When trying to this library with a complex json. for example:

Attempting to match.Any("#.components.#.properties.value") yields a bad snapshot.
modifying to match.Any("0.components.#.properties.value") yields a good but incomplete matched snapshot.

A solution for prerendering the matchers based on the input is non-functional. (creating a matcher per nested of nested matcher).

It's probably due to limitations with gjson - tidwall/gjson#267
As you figured, snapshots can be (and probably are) complex, nested arrays within an array is very common, this yields using a snapshot almost useless.

We can try fixing it ourselves, parsing the '.#.' and running through a loop with '.0.' ... '.n.' new matchers to workaround this.
wdyt?

[TestGenerateSync/happy:cyclonedx_mono_with_and_go_trivy - 1]
{
 "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
 "bomFormat": "CycloneDX",
 "components": [
  {
   "bom-ref": "<Any value>",
   "name": "go.mod",
   "properties": [
    {
     "name": "aquasecurity:trivy:Class",
     "value": "lang-pkgs"
    },
    {
     "name": "aquasecurity:trivy:Type",
     "value": "gomod"
    }
   ],
   "type": "application"
  },
  {
   "bom-ref": "<Any value>",
   "name": "github.com/davecgh/go-spew",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/davecgh/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/davecgh/[email protected]",
   "type": "library",
   "version": "1.1.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "github.com/jmespath/go-jmespath/internal/testify",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/jmespath/go-jmespath/internal/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/jmespath/go-jmespath/internal/[email protected]",
   "type": "library",
   "version": "1.5.1"
  },
  {
   "bom-ref": "<Any value>",
   "licenses": [
    {
     "license": {
      "name": "Apache-2.0"
     }
    }
   ],
   "name": "github.com/jmespath/go-jmespath",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/jmespath/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/jmespath/[email protected]",
   "type": "library",
   "version": "0.4.0"
  },
  {
   "bom-ref": "<Any value>",
   "licenses": [
    {
     "license": {
      "name": "BSD-2-Clause"
     }
    }
   ],
   "name": "github.com/pkg/errors",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/pkg/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/pkg/[email protected]",
   "type": "library",
   "version": "0.9.1"
  },
  {
   "bom-ref": "<Any value>",
   "name": "github.com/pmezard/go-difflib",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/pmezard/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/pmezard/[email protected]",
   "type": "library",
   "version": "1.0.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "github.com/stretchr/objx",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/stretchr/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/stretchr/[email protected]",
   "type": "library",
   "version": "0.1.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "github.com/yuin/goldmark",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "github.com/yuin/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/github.com/yuin/[email protected]",
   "type": "library",
   "version": "1.4.13"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/crypto",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.0.0-20210921155107-089bfa567519"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/mod",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.6.0-dev.0.20220419223038-86c51ed26bb4"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/net",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.1.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/sync",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.0.0-20220722155255-886fb9371eb4"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/sys",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.1.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/term",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.1.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/text",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.4.0"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/tools",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.1.12"
  },
  {
   "bom-ref": "<Any value>",
   "name": "golang.org/x/xerrors",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "golang.org/x/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/golang.org/x/[email protected]",
   "type": "library",
   "version": "0.0.0-20190717185122-a985d3407aa7"
  },
  {
   "bom-ref": "<Any value>",
   "name": "gopkg.in/check.v1",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "gopkg.in/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/gopkg.in/[email protected]",
   "type": "library",
   "version": "0.0.0-20161208181325-20d25e280405"
  },
  {
   "bom-ref": "<Any value>",
   "name": "gopkg.in/yaml.v2",
   "properties": [
    {
     "name": "aquasecurity:trivy:PkgID",
     "value": "gopkg.in/[email protected]"
    },
    {
     "name": "aquasecurity:trivy:PkgType",
     "value": "gomod"
    }
   ],
   "purl": "pkg:golang/gopkg.in/[email protected]",
   "type": "library",
   "version": "2.2.8"
  }
 ],
 "dependencies": [
  {
   "dependsOn": [
    "a0fedf37-f446-4629-ab6c-fbebd72d5034"
   ],
   "ref": "4dacd68b-b238-4db3-a1b1-d388034af718"
  },
  {
   "dependsOn": [
    "pkg:golang/github.com/davecgh/[email protected]",
    "pkg:golang/github.com/jmespath/[email protected]",
    "pkg:golang/github.com/pkg/[email protected]",
    "pkg:golang/github.com/pmezard/[email protected]",
    "pkg:golang/github.com/stretchr/[email protected]",
    "pkg:golang/github.com/yuin/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/golang.org/x/[email protected]",
    "pkg:golang/gopkg.in/[email protected]",
    "pkg:golang/gopkg.in/[email protected]"
   ],
   "ref": "a0fedf37-f446-4629-ab6c-fbebd72d5034"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/davecgh/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/jmespath/go-jmespath/internal/[email protected]"
  },
  {
   "dependsOn": [
    "pkg:golang/github.com/jmespath/go-jmespath/internal/[email protected]"
   ],
   "ref": "pkg:golang/github.com/jmespath/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/pkg/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/pmezard/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/stretchr/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/github.com/yuin/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/golang.org/x/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/gopkg.in/[email protected]"
  },
  {
   "dependsOn": [],
   "ref": "pkg:golang/gopkg.in/[email protected]"
  }
 ],
 "metadata": {
  "component": {
   "bom-ref": "<Any value>",
   "name": "testdata/aws-sdk-go",
   "properties": [
    {
     "name": "aquasecurity:trivy:SchemaVersion",
     "value": "2"
    }
   ],
   "type": "application"
  },
  "timestamp": "<Any value>",
  "tools": [
   {
    "name": "trivy",
    "vendor": "aquasecurity",
    "version": "0.47.0"
   }
  ]
 },
 "serialNumber": "<Any value>",
 "specVersion": "1.5",
 "version": 1,
 "vulnerabilities": []
}
---

Steps to Reproduce

Run with attached JSON (or simplified version) or any [{ ... , arr: [ { id: 123 } ]}] with a "#.arr.#.id" matcher.

Expected Behavior

No response
@manifestori manifestori added the bug Something isn't working label Nov 26, 2023
@gkampitakis
Copy link
Owner

Hey 👋 thanks a lot for opening this issue and using the library. I am not super familiar with the more complicated cases of gjson syntax, I have only used the simple examples.

So for me to understand the issue with #.arr.#.id where doesn't iterrate through all the items? does this happen only for double nested? or triple nested? What was your idea for handling this in go-snaps level?

@manifestori
Copy link
Author

Hey @gkampitakis it doesn't matter if it's double or triple. If you use # more than once, you get a false positive match and the snapshot will yield just a <Any value> result. (unusable snapshot).

My idea was to parse the path before passing it to gjson. Breaking it down to multiple calls to gjson (instead of 1 call with #.prop.#.id), pass n times 0...n.prop.# to gjson and merge result (recursively)

this should be fixed on Jason, but maybe it's an easier workaround to implement.

@gkampitakis
Copy link
Owner

Hey 👋 I am not sure how much I want to add this fix in go-snaps 😞

@gkampitakis gkampitakis added the wontfix This will not be worked on label Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gkampitakis @manifestori