Schema validation seems to ignore formats

[tschmidtb51]

It looks like the current version (3.0.0-beta1; but it might have existed in previous version), does not evaluated the format constrains. This was discovered when checking https://github.com/cisagov/CSAF/blob/971b6976d1ce52a710025aef920dba72d3a0675e/csaf_files/OT/white/2023/icsa-23-159-02.json#L269 resulted in a different amount of errors while using the csaf_validator with the csaf_validator-service preset of mandatory (0) vs basic (1).

We need to find out, why the uri wasn't checked for the format.

[s-l-teichmann]

Turns out that the Schema Compiler from the validation library we use has a flag to check the formats.
It is off by default.

PR #475 turns this flag on.

Without the PR:

./csaf_validator icsa-23-159-02.json
"icsa-23-159-02.json" passes the schema validation.

With the PR:

./csaf_validator icsa-23-159-02.json
schema validation errors of "icsa-23-159-02.json"
  * https://docs.oasis-open.org/csaf/csaf/v2.0/csaf_json_schema.json#: doesn't validate with https://docs.oasis-open.org/csaf/csaf/v2.0/csaf_json_schema.json#
  * /vulnerabilities/0/remediations/3/url: 'www.illustracameras.com' is not valid 'uri'

I'm not totally sure what the overall consequences of always turning on this flag are but I think we should go with it.

[tschmidtb51]

This seem not to be resolved for the following file:

{
  "document": {
    "category": "csaf_base",
    "csaf_version": "2.0",
    "publisher": {
      "category": "other",
      "name": "Test",
      "namespace": "https://www.example.com"
    },
    "references": [
      {
        "summary": "Invalid URL",
        "url": "https://security.business.xerox.com/wp-content/uploads/2022/11/Xerox-Security-Bulletin-XRX22-026-FreeFlow®-Print-Server-v7.pdf"
      }
    ],
    "title": "Testfile with invalid URI",
    "tracking": {
      "current_release_date": "2023-11-30T12:25:34.622Z",
      "generator": {
        "date": "2023-11-30T12:27:08.481Z",
        "engine": {
          "name": "Secvisogram",
          "version": ".2.2.15"
        }
      },
      "id": "Test-2023-11-30",
      "initial_release_date": "2023-11-30T12:25:34.622Z",
      "revision_history": [
        {
          "date": "2023-11-30T12:25:34.622Z",
          "number": "1",
          "summary": "Initial version."
        }
      ],
      "status": "draft",
      "version": "1"
    }
  }
}

[s-l-teichmann]

Saved as test-2023-11-30.json

./csaf_validator -o short --validator=http://0.0.0.0:8082 --validator_preset=basic test-2023-11-30.json
"test-2023-11-30.json" passes the schema validation.
isValid: false
errors:
  message: must match format "uri"
  instance path(s):
    /document/references/0/url
"test-2023-11-30.json" does not pass remote validation.

[tschmidtb51]

@s-l-teichmann Thanks for adding the output. To be precise: I would expect that the file also fails the Go schema validation.

[s-l-teichmann]

uri format validation is table driven in https://github.com/santhosh-tekuri/jsonschema/blob/9e31c47bfa9dfdb5ee04dec04fab51117e43c45a/format.go#L30 with the implementation
in https://github.com/santhosh-tekuri/jsonschema/blob/9e31c47bfa9dfdb5ee04dec04fab51117e43c45a/format.go#L390

Checked out https://github.com/santhosh-tekuri/jsonschema
Modified format_test.go:

diff --git a/format_test.go b/format_test.go
index b8fdded..018890a 100644
--- a/format_test.go
+++ b/format_test.go
@@ -238,6 +238,7 @@ func TestIsIPV6(t *testing.T) {
 
 func TestIsURI(t *testing.T) {
        tests := []test{
+               {"https://foobar®.pdf", false},
                {"http://foo.bar/?baz=qux#quux", true},
                {"//foo.bar/?baz=qux#quux", false}, // an invalid protocol-relative URI Reference
                {"\\\\WINDOWS\\fileshare", false},  // an invalid URI

Run

go test -run TestIsURI                                                         master!
--- FAIL: TestIsURI (0.00s)
    format_test.go:249: #0: "https://foobar®.pdf", valid false, got valid true
FAIL
exit status 1
FAIL	github.com/santhosh-tekuri/jsonschema/v5	0.007s

So the isUri does not do the expected.