Improve compile performance

[StarpTech]

Hi, first of all thanks for the great library. We're quite happy with it so far. We are experiencing slow compile performance for a small schema. Due to this we already switched the approach for our serverless router to avoid any additional cold start latency.

Compile: ~36ms
Validation: ~318µs

Version: 5

Test case: https://github.com/wundergraph/cosmo/blob/main/router/pkg/config/json_schema.go#L203
JSON Schema: https://github.com/wundergraph/cosmo/blob/main/router/pkg/config/config.schema.json

We didn't investigate on this yet but maybe @santhosh-tekuri you already know more.

Hardware: Mac M3 Max

[santhosh-tekuri]

You don't need compile schema each time you want to validate. You can compile it once and using it for each request to validate.

You said Compile took 16ms. Have you excluded json parsing time. Compiler.AddResource does the json parsing.

You can update jsonschema library from v5 to v6. But I don't expect any drastic change in performance though.
I you are ok, I can check how it performs in v6.

[santhosh-tekuri]

I have benchmarked it with v6 and compile took 2857313 ns i.e 2.8ms

below is the benchmark results for three consecutive runs on M1 Max 64GB:

$ go test -bench=.
goos: darwin
goarch: arm64
pkg: github.com/santhosh-tekuri/jsonschema/v6
BenchmarkCompile-10          396           2857313 ns/op
PASS
ok      github.com/santhosh-tekuri/jsonschema/v6        2.599s

$ go test -bench=.
goos: darwin
goarch: arm64
pkg: github.com/santhosh-tekuri/jsonschema/v6
BenchmarkCompile-10          396           2844803 ns/op
PASS
ok      github.com/santhosh-tekuri/jsonschema/v6        2.132s

$ go test -bench=.
goos: darwin
goarch: arm64
pkg: github.com/santhosh-tekuri/jsonschema/v6
BenchmarkCompile-10          400           2865170 ns/op
PASS
ok      github.com/santhosh-tekuri/jsonschema/v6        2.085s

below is the benchmark source code(you can try to run this on your laptop and check):

package jsonschema_test

import (
	"log"
	"os"
	"testing"

	"github.com/santhosh-tekuri/jsonschema/v6"
)

func BenchmarkCompile(b *testing.B) {
	f, err := os.Open("./config.schema.json")
	if err != nil {
		log.Fatal(err)
	}
	schema, err := jsonschema.UnmarshalJSON(f)
	if err != nil {
		b.Fatal(err)
	}
	url := "http://temp.com/config.json" // use valid url to improve performance

	for i := 0; i < b.N; i++ {
		c := jsonschema.NewCompiler()
		if err := c.AddResource(url, schema); err != nil {
			b.Fatal(err)
		}
		_, err = c.Compile(url)
		if err != nil {
			b.Fatal(err)
		}
	}
}

[santhosh-tekuri]

you can use valid url in your code which may improve performance:

change:

        err := c.AddResource("config.schema.json", strings.NewReader(schema))
	if err != nil {
		return err
	}

	sch, err := c.Compile("config.schema.json")

to

        url := "http://temp.com/config.schema.json"
        err := c.AddResource(url, strings.NewReader(schema))
	if err != nil {
		return err
	}

	sch, err := c.Compile(url)

[StarpTech]

It had no effect.

You said Compile took 16ms. Have you excluded json parsing time. Compiler.AddResource does the json parsing.

Yes, I test from jsonschema.NewCompiler() to validate.

You can update jsonschema library from v5 to v6. But I don't expect any drastic change in performance though.
I you are ok, I can check how it performs in v6.

I updated the issue description. I tested v6 instead of v5. V5 took around 37ms. I think 16ms is still a lot for such a small config. What do you think?

[santhosh-tekuri]

The testcase you have provided is using v5. and in v5 there is no way to exclude json parsing. the line json_schema.go:222 internally does json parsing.

so in your test the time takes is including the time taken to parse the json.

if you are using v6, you can do json parsing outside the library. that is what I am doing in the benchmark test I provided in previous comment. and it just takes 2.8ms to compile

have you tested the the benchmark I provided at your end. please test that benchmark and check the time yourself

[StarpTech]

Got it, I thought you were asking for something different. With v6, I tested only compilation. From compiler instantiation and compiling the schema, it takes 17ms. I benchmarked it and produced very similar numbers. What this test doesn't consider are one-off shots. It is probably coupled to mem alloc.

[santhosh-tekuri]

Did you try the benchmark I provided. It takes only 2.8 ms. May be your one off shots taking time. Can you provide benchmark that can reproduce your results