feat: new evaluation routes (#1824)

* Feat: eval v2 sqlite (#1739) * feat(wip): rollout for boolean flags * feat(wip): strategy impl * feat: rollout rules > rollout strategy * chore: add namespace_key to tables * feat: list rules * feat: impl delete * chore: rm debug logs * chore: rename things * chore: compiles, start adding storage tests * feat(wip): update rollout * chore: compiles * chore: compiles/finish sqlite rollout tests for now * chore: fix countRollouts call * chore: rebase eval-v2 * chore: checkout ui from main * chore: replace ui with ui from main * chore: fix buflint * chore: fix lint errors * chore: rowserrcheck * chore: mage proto * feat: implement gRPC server for the Variant RPC method * chore: rename Server field to evaluator * chore: move around tests, and other fixes * chore: DRY up middleware and add extra field to otel metrics * feat: add boolean evaluation logic for the store * chore: add tests for boolean evaluate and address PR comments from upstream * chore: add batch evaluation functionality * chore: address comments and add more tests for behavior * feat: implement gRPC server for the Variant RPC method (#1798) * feat: implement gRPC server for the Variant RPC method * chore: rename Server field to evaluator * chore: move around tests, and other fixes * chore: DRY up middleware and add extra field to otel metrics * test(integration): ensure Evaluation.Variant returns as expected (#1800) * test(integration/api): ensure Evaluation.Variant returns as expected * test(integration/readonly): ensure Evaluation.Variant returns as expected * fix(build/testing): set production yaml version to 1.0 * fix(build/testing/api): ensure disabled flag is created and listed * fix(build/testing/api): fix disabled flag asserts and delete before ns is deleted * fix(build/testdata): typo on name field case * fix(server/middleware): set request IDs on evaluation req/resp type * fix(build/integration): ensure 51 flags exist * fix(build/integration): ensure flag not found error is returned * fix(build/integration): explicitly set enabled false in testdata * chore: rearrange some files and make some new declarations * chore: move around some structures and address other feedback around passing flag into Evaluate * chore: set timestamps and request id on Variant responses * chore: add test and move invalid flag type to evaluator --------- Co-authored-by: George <[email protected]> * chore: readd tests and other couple fixes * chore: add middleware bits for timestamp and duration * chore: remove storage implementation of getting rollouts * chore: use lowercase package names, and return nil instead of real value * chore: add tests for coverage * chore: add error response for batch evaluation * fix(api/rollouts): ensure tx rollback (#1805) * test(integration/api): ensure Boolean type flag create * fix(api/rollouts): ensure tx rollback and drop create rollout type * fix(storage/rollout): ensure tx rollback and fixed rollout type on update * fix(storage/sql/rollout): validate rule is supplied on create * chore(rpc): add rollout request types validation * fix(integration/readonly): add flag_disabled to end of set * chore: commit proto version comment updates * chore: use different enum for batch evaluation responses * chore: make small optimization for struct initialization * refactor!(rpc/flipt): rename RolloutPercentage to RolloutThreshold * fix(tests): change assertions to match threshold not percentage enum string * refactor(storage): rename RolloutPercentage to RolloutThreshold * chore: change proto definition for evaluation and error evaluation reasons * chore: implement storage queries for rollout evaluation * chore: add integration test for boolean, batch * feat(ui): add flag type radio (#1830) * feat(wip): flag type select/render * chore: make disabled on update * chore: fix default checked * chore: address PR comments * feat(ext): add support for flag type and rollout definitions (#1826) * test(ext): add failing cases for new version 1.1 and rollouts * feat(ext): add support for flag type and rollout definitions * fix(ext): adding missing error not-nil check * test(readonly): bump testdata import version * fix(ext): support rename from rollout percentage to threshold * fix(ext): ensure minimum version enforced as expected * test(readonly): update suite with boolean flag assertions * feat(ext): support implicit rank in flag state version 1.1 * chore(ext): correct typos in comments * fix(test/cli): update assertion for import/export * feat(validate): add support for state version 1.1 * chore(build): address comment fix suggestions * fix(test/migration): use an import file valid for latest * fix(test/migration): import using latest build not tip * feat: add fs storage implentation and readonly tests * feat: address all the missing links with rollouts in regards to fs store * feat: create eval rollout threshold (#1835) * feat(wip): ui rollout threshold crud * feat(wip): show new rollout button * chore: add type to flag table * feat(wip): threshold rule form * feat(wip): rollout thresholds * chore: rename to 'rules' for now * chore: rm un-needed prop * chore: fix small issues * chore: extract some more constants * feat: add missing description, set max/min for threshold client side * fix: typing * chore: type the form values * chore: move value to common form * chore: disable evaluation tab for boolean flag * chore(fs/rollouts): address PR comments and add relevant tests * fix: broken ui build (#1838) * feat: add audit logging for rollout actions (#1834) * feat: add audit logging for rollout actions * chore(tests): add unit tests for delete in the middleware * feat: eval rollout segment ui (#1839) * feat(wip): start * feat(ui): finish segment rollout creation form * fix(rollouts): add order rollouts and validate requests (#1850) * feat(rollouts): add order rollouts RPC * fix(rollouts): implement server.OrderRollouts and add unit tests * fix(proto): add gateway routes for order rollouts * test(readonly): ensure list rollout pagination * chore(rpc/flipt): add validation around create and update rollouts * fix(ext): add variant description to testdata yaml * fix(build/testing/migration): use previous release versions test suite (#1852) * fix(build/testing/migration): use previous release versions test suite * refactor(build/testing/migration): use readonly test suite data from latest release * feat(middleware): support caching new evaluation RPCs (#1853) * feat(middleware): support caching evaluation.VariantEvaluationResponse * feat(middleware): support caching evaluation.BooleanEvaluationResponse * feat(eval/rollouts): write benchmark tests for evaluations * chore(benchmarks): move around logic * feat(auth): ability to skip authentication for top-level api prefixes (#1854) * feat(auth): ability to skip authentication for top-level api prefixes * fix(config): add mapstructure tags * fix(cmd/grpc): thread authentication options * Rule List refactor (#1855) * feat: redo rules list styles * feat: done tweaking the ui * chore: drag drop * chore: show actual variant key; fix border color on drag * chore: make responsive * chore(sql): format query * feat: real rollout list/quick edit/delete (#1857) * feat(wip): rollout list ui + quickedit * chore: update some styles * chore: fix selected segment * feat: finish update/delete of rollout list * chore: fix linter * chore: add percent icon, hide slider in mobile * feat: make reset work: * chore: clean up benchmarks to not have random elements in the requests * feat(console-evaluation): use v2 evaluation in console rather than v1 evaluation * feat: rollout edit + typescript enum redo (#1859) * chore: address comments * feat: Quick edit rules (#1862) * feat(metrics): Add otel and general metrics for boolean evaluation * chore: pass in context from the outside for boolean match * fix(build): use relative replace for rpc/flipt (#1881) * fix(build): use relative replace for rpc/flipt * chore(build): replace root with relative directory * chore: profiling via Dagger (#1873) * chore: rebase on eval-v2 * chore: rm binary * feat(wip): attempt to run profiling / pyroscope in dagger * chore: go.sum things * chore: more mod tidy * chore: give proper user * chore: finish profile run * chore: finish loadtest via dagger * chore: bump to 60s, add more to README * chore: fix loadtest to actually import / use state * chore: rm un-needed export * feat(ui): eval default rollout (#1866) * chore: change boolean eval to use false as default * chore: check for default namespace * chore: change back to use enabled as default, rename boolean eval response field * chore: return reason from matchConstraints * feat: add default rollout in UI * chore: fix api IT * chore(sqlite): add custom logic for sqlite for returning errors on specific cases * feat(db): Add the rest of sql files for rollout migrations * chore: runs prettier after dependency update (#1883) * chore: go mod tidy * chore(mysql): address comments and make query squirrel builder * chore(mysql): disable sql_mode for testing * chore: change property of Value to Enabled * chore: add default value to current_timestamp as well * chore: remove unique index * fix: validation of rollout rule values in forms (#1889) * chore: rollouts UI its (#1888) * chore: small ui tweaks for rules/rollouts (#1890) * chore: prettier package.json * chore: add namespace key to error reason; rename error enum (#1902) * chore: boolean eval screenshots (#1904) * chore: fix screenshots for new ui * chore: mv screenshots to subdir * feat(wip): concepts docs * chore: rm old screenshots func * chore: finish most of the screenshots * chore: add rollouts * chore: add readonly screenshot * chore: add missing eval console screen * chore: fix scrollToBottom for screenshots (#1911) * chore: fix scrollToBottom for screenshots * chore: increase sleep by 1s * chore: mod tidy * chore: loadtest cleanup * chore: rework interceptor config * chore: fix cacher interceptor ordering (#1914) * chore: fix cacher interceptor ordering * chore: fix storage cacher tests (#1916) * chore: why are you changing go.work.sum * chore: fixup cacher tests into new eval-v2 branch * chore: rm unknown file --------- Co-authored-by: Mark Phelps <[email protected]> Co-authored-by: Yoofi Quansah <[email protected]>
flipt-io · Jul 28, 2023 · 8bb0e95 · 8bb0e95
1 parent 3904c27
commit 8bb0e95
Show file tree

Hide file tree

Showing 253 changed files with 36,884 additions and 12,449 deletions.
diff --git a/.gitignore b/.gitignore
@@ -39,8 +39,7 @@ __debug_bin
 
 Brewfile.lock.json
 *.key
-.task/
-.build/
+build/hack/out/
 examples/cockroachdb/data
 playwright-report/
 screenshots/
diff --git a/.golangci.yml b/.golangci.yml
@@ -78,5 +78,6 @@ linters-settings:
     excludes: # exclude gosec messages about weak crypto
       - "G501"
       - "G401"
+      - "G404"
   goconst:
     ignore-tests: true
diff --git a/build/README.md b/build/README.md
@@ -73,3 +73,25 @@ These tests exercise the Flipt Go SDK against a matrix of Flipt configurations.
 ### CLI Tests
 
 `mage test:cli` runs a suite of [CLI tests](./testing/cli.go) invoking the `flipt` binary and its subcommands.
+
+## Hack
+
+The `hack` namespace within the Mage targets can be used to run various tasks that are under active development.
+
+```sh
+  hack:loadTest       runs a load test against a running instance of Flipt using Pyroscope and vegeta.
+```
+
+### Load Test
+
+`mage hack:loadTest` runs a load test against a running instance of Flipt using [Pyroscope](https://pyroscope.io) and vegeta.
+
+The test will import data from [testdata/default.yaml](testing/integration/readonly/testdata/default.yaml) into Flipt and then run a series of evaluation requests against a running instance of Flipt for a period of 60 seconds.
+
+After running this command, the results will be available in the `./build/hack/out/profiles` directory.
+
+You can view the results using the Pyroscope UI by running `pyroscope server --adhoc-data-path="$(pwd)/hack/out/profiles"`. 
+
+This will start the Pyroscope server on `http://localhost:4040`. 
+
+**Note:** You will need to have Pyroscope installed locally to run this command (See [Pyroscope Quick Start](https://pyroscope.io/docs/server-install-macos/)). (TODO: run this in a container)
diff --git a/build/generate/screenshots.go b/build/generate/screenshots.go
@@ -0,0 +1,135 @@
+package generate
+
+import (
+	"context"
+	"crypto/sha256"
+	"fmt"
+	"log"
+	"os"
+	"path"
+	"time"
+
+	"dagger.io/dagger"
+	"golang.org/x/sync/errgroup"
+)
+
+func Screenshots(ctx context.Context, client *dagger.Client, flipt *dagger.Container) error {
+	src := client.Host().Directory("./ui/", dagger.HostDirectoryOpts{
+		Include: []string{
+			"./package.json",
+			"./package-lock.json",
+			"./playwright.config.ts",
+			"/screenshots/",
+		},
+	})
+
+	contents, err := src.File("package-lock.json").Contents(ctx)
+	if err != nil {
+		return err
+	}
+
+	cache := client.CacheVolume(fmt.Sprintf("node-modules-screenshot-%x", sha256.Sum256([]byte(contents))))
+
+	ui, err := client.Container().From("node:18-bullseye").
+		WithMountedDirectory("/src", src).WithWorkdir("/src").
+		WithMountedCache("/src/node_modules", cache).
+		WithExec([]string{"npm", "install"}).
+		WithExec([]string{"npx", "playwright", "install", "chromium", "--with-deps"}).
+		Sync(ctx)
+	if err != nil {
+		return err
+	}
+
+	src = client.Host().Directory("./ui/", dagger.HostDirectoryOpts{
+		Exclude: []string{
+			"./dist/",
+			"./node_modules/",
+		},
+	})
+
+	// remount entire directory with module cache
+	ui, err = ui.WithMountedDirectory("/src", src).
+		WithMountedCache("/src/node_modules", cache).
+		WithExec([]string{"npm", "install"}).
+		Sync(ctx)
+	if err != nil {
+		return err
+	}
+
+	dirs := []string{
+		"getting_started", "concepts", "configuration", "extra",
+	}
+
+	for _, dir := range dirs {
+		var (
+			g          errgroup.Group
+			containers = make(chan *dagger.Container)
+			dir        = dir
+		)
+
+		entries, err := ui.Directory("screenshot/" + dir).Entries(ctx)
+		if err != nil {
+			// skip if directory does not exist
+			continue
+		}
+
+		go func() {
+			_ = g.Wait()
+			close(containers)
+		}()
+
+		for _, entry := range entries {
+			entry := entry
+			g.Go(func() error {
+				test, err := buildUI(ctx, ui, flipt)
+				if err != nil {
+					return err
+				}
+
+				if ext := path.Ext(entry); ext != ".js" {
+					return nil
+				}
+
+				c, err := test.WithExec([]string{"node", path.Join("screenshot", dir, entry)}).Sync(ctx)
+				if err != nil {
+					return err
+				}
+
+				containers <- c
+				log.Printf("Generating screenshot for %s/%s\n", dir, entry)
+
+				return err
+			})
+		}
+
+		for c := range containers {
+			if _, err := c.Directory("screenshots").
+				Export(ctx, "screenshots"); err != nil {
+				return err
+			}
+		}
+	}
+
+	return err
+}
+
+func buildUI(ctx context.Context, ui, flipt *dagger.Container) (_ *dagger.Container, err error) {
+	flipt, err = flipt.Sync(ctx)
+	if err != nil {
+		return nil, err
+	}
+
+	ui, err = ui.Sync(ctx)
+	if err != nil {
+		return nil, err
+	}
+
+	return ui.
+		WithServiceBinding("flipt", flipt.
+			WithEnvVariable("CI", os.Getenv("CI")).
+			WithEnvVariable("FLIPT_AUTHENTICATION_METHODS_TOKEN_ENABLED", "true").
+			WithEnvVariable("UNIQUE", time.Now().String()).
+			WithExec(nil)).
+		WithFile("/usr/bin/flipt", flipt.File("/flipt")).
+		WithEnvVariable("FLIPT_ADDRESS", "http://flipt:8080"), nil
+}
diff --git a/build/go.mod b/build/go.mod
@@ -9,21 +9,24 @@ require (
 	github.com/aws/aws-sdk-go-v2/config v1.18.28
 	github.com/aws/aws-sdk-go-v2/service/s3 v1.37.0
 	github.com/containerd/containerd v1.6.19
-	github.com/docker/docker v23.0.3+incompatible
+	github.com/docker/docker v23.0.5+incompatible
 	github.com/go-git/go-billy/v5 v5.4.1
-	github.com/go-git/go-git/v5 v5.6.1
+	github.com/go-git/go-git/v5 v5.7.0
 	github.com/gofrs/uuid v4.4.0+incompatible
 	github.com/google/go-cmp v0.5.9
+	github.com/google/go-github/v53 v53.2.0
 	github.com/google/uuid v1.3.0
-	github.com/magefile/mage v1.14.0
+	github.com/magefile/mage v1.15.0
 	github.com/opencontainers/image-spec v1.1.0-rc2
-	github.com/stretchr/testify v1.8.2
-	go.flipt.io/flipt v1.19.1
-	go.flipt.io/flipt/sdk/go v0.0.0-00010101000000-000000000000
+	github.com/stretchr/testify v1.8.4
+	github.com/tsenart/vegeta v12.7.0+incompatible
+	go.flipt.io/flipt v1.23.3
+	go.flipt.io/flipt/rpc/flipt v1.22.0
+	go.flipt.io/flipt/sdk/go v0.3.0
 	golang.org/x/mod v0.12.0
 	golang.org/x/sync v0.3.0
-	google.golang.org/grpc v1.55.0
-	google.golang.org/protobuf v1.30.0
+	google.golang.org/grpc v1.56.1
+	google.golang.org/protobuf v1.31.0
 	gopkg.in/yaml.v2 v2.4.0
 	sigs.k8s.io/kind v0.17.0
 )
@@ -32,8 +35,8 @@ require (
 	github.com/99designs/gqlgen v0.17.34 // indirect
 	github.com/BurntSushi/toml v1.2.1 // indirect
 	github.com/Khan/genqlient v0.6.0 // indirect
-	github.com/Microsoft/go-winio v0.6.0 // indirect
-	github.com/ProtonMail/go-crypto v0.0.0-20230217124315-7d5c6f04bbb8 // indirect
+	github.com/Microsoft/go-winio v0.6.1 // indirect
+	github.com/ProtonMail/go-crypto v0.0.0-20230710112148-e01326fd72eb // indirect
 	github.com/acomagu/bufpipe v1.0.4 // indirect
 	github.com/adrg/xdg v0.4.0 // indirect
 	github.com/alessio/shellescape v1.4.1 // indirect
@@ -52,53 +55,69 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.13 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sts v1.19.3 // indirect
 	github.com/aws/smithy-go v1.13.5 // indirect
+	github.com/bmizerany/perks v0.0.0-20230307044200-03f9df79da1e // indirect
 	github.com/cloudflare/circl v1.3.3 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/dgryski/go-gk v0.0.0-20200319235926-a69029f61654 // indirect
 	github.com/docker/distribution v2.8.2+incompatible // indirect
 	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/emirpasic/gods v1.18.1 // indirect
 	github.com/evanphx/json-patch/v5 v5.6.0 // indirect
-	github.com/go-git/gcfg v1.5.0 // indirect
+	github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/google/go-querystring v1.1.0 // indirect
 	github.com/google/safetext v0.0.0-20220905092116-b49f7bc46da2 // indirect
 	github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
 	github.com/hashicorp/go-version v1.2.1 // indirect
 	github.com/iancoleman/strcase v0.3.0 // indirect
-	github.com/imdario/mergo v0.3.13 // indirect
+	github.com/imdario/mergo v0.3.15 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/influxdata/tdigest v0.0.1 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
 	github.com/kevinburke/ssh_config v1.2.0 // indirect
-	github.com/kr/pretty v0.3.1 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/matryer/is v1.4.0 // indirect
 	github.com/mattn/go-isatty v0.0.19 // indirect
-	github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/pelletier/go-toml v1.9.5 // indirect
 	github.com/pjbgf/sha1cd v0.3.0 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
-	github.com/rogpeppe/go-internal v1.10.0 // indirect
 	github.com/sergi/go-diff v1.3.1 // indirect
-	github.com/sirupsen/logrus v1.9.0 // indirect
-	github.com/skeema/knownhosts v1.1.0 // indirect
+	github.com/sirupsen/logrus v1.9.2 // indirect
+	github.com/skeema/knownhosts v1.1.1 // indirect
 	github.com/spf13/cobra v1.7.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
+	github.com/streadway/quantile v0.0.0-20220407130108-4246515d968d // indirect
 	github.com/vektah/gqlparser/v2 v2.5.6 // indirect
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
+	go.flipt.io/flipt/errors v1.19.3 // indirect
+	go.uber.org/atomic v1.11.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	go.uber.org/zap v1.24.0 // indirect
 	golang.org/x/crypto v0.11.0 // indirect
 	golang.org/x/net v0.12.0 // indirect
+	golang.org/x/oauth2 v0.8.0 // indirect
 	golang.org/x/sys v0.10.0 // indirect
 	golang.org/x/text v0.11.0 // indirect
-	golang.org/x/time v0.3.0 // indirect
 	golang.org/x/tools v0.11.0 // indirect
-	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/genproto v0.0.0-20230526203410-71b5a4ffd15e // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect
 	gopkg.in/warnings.v0 v0.1.2 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	gotest.tools/v3 v3.1.0 // indirect
 	sigs.k8s.io/yaml v1.3.0 // indirect
 )
 
-replace go.flipt.io/flipt/sdk/go => ../sdk/go
+replace (
+	go.flipt.io/flipt/ => ../
+	go.flipt.io/flipt/rpc/flipt => ../rpc/flipt
+	go.flipt.io/flipt/sdk/go => ../sdk/go
+)