Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(segments): Segment AND'ing #1915

Merged
merged 57 commits into from
Aug 15, 2023
Merged

feat(segments): Segment AND'ing #1915

merged 57 commits into from
Aug 15, 2023

Conversation

yquansah
Copy link
Contributor

@yquansah yquansah commented Jul 27, 2023

Segment AND'ing project.

Supporting the ability for users to add multiple segments to their rules/rollouts for evaluations.

Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple quick thoughts

rpc/flipt/flipt.proto Outdated Show resolved Hide resolved
rpc/flipt/flipt.proto Outdated Show resolved Hide resolved
rpc/flipt/flipt.proto Outdated Show resolved Hide resolved
rpc/flipt/flipt.proto Show resolved Hide resolved
rpc/flipt/flipt.proto Outdated Show resolved Hide resolved
rpc/flipt/flipt.proto Outdated Show resolved Hide resolved
config/migrations/sqlite3/11_segment_anding.up.sql Outdated Show resolved Hide resolved
Base automatically changed from eval-v2 to main July 28, 2023 12:22
@yquansah yquansah changed the title feat(segments): POC of API parity with segment anding feat(segments): Segment AND'ing Jul 28, 2023
@yquansah yquansah marked this pull request as ready for review July 31, 2023 17:17
@yquansah yquansah requested a review from a team as a code owner July 31, 2023 17:17
Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theres a few lint failures around err checking that need 👀 I think, also wondering how performance is impacted with this change

internal/ext/importer.go Show resolved Hide resolved
internal/server/evaluation/evaluation.go Outdated Show resolved Hide resolved
internal/storage/fs/snapshot.go Outdated Show resolved Hide resolved
internal/storage/sql/common/evaluation.go Outdated Show resolved Hide resolved
internal/storage/sql/common/rollout.go Outdated Show resolved Hide resolved
internal/storage/sql/common/rollout.go Outdated Show resolved Hide resolved
internal/storage/sql/common/rollout.go Outdated Show resolved Hide resolved
internal/storage/sql/common/rule.go Outdated Show resolved Hide resolved
rpc/flipt/flipt.proto Show resolved Hide resolved
internal/storage/storage.go Outdated Show resolved Hide resolved
* feat(update): UI for Rule updates

* feat(rollouts/rules): Updating UI for rollouts and rules

* feat(update): UI for Rule updates

* feat: Add readonly attributes to edit forms

* chore: address comments

* chore: fix logic for updating
@yquansah yquansah marked this pull request as ready for review August 8, 2023 15:08
* chore: fix reset of rules/rollouts

* chore: fix null error with segment
…nd UX around adding segments (#1975)

* feat(rules/rollouts): hide or and and based on segment keys length, and change UX for adding segments

* chore: fix UI integration tests
* feat(rules): Make segment be one of two types

* chore: force segment operator to be OR for one segment
* chore: print statements

chore: revise query

chore: remove unnecessary lines

* chore: defer close ruleMetaRows

* chore: remove orderBy since we are sorting at the end of the function

* chore: fix double quotes for rank for MySQL support
Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one minor json comment for struct tags, thank you for all the hard work on this!!

SegmentOperator flipt.SegmentOperator `json:"segmentOperator,omitempty"`
}

type EvaluationSegment struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably put the JSON struct tags on this just to be safe since this will also be cached

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markphelps Thank you for this. I could have just committed directly to this now that I think about it. Here is the PR: #1984

Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets do it to it!

Copy link
Contributor

@GeorgeMac GeorgeMac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epic work 💪 Couple optimization which are take it or leave it. Either way merge away when ready 👍

From(tableRolloutSegments).
Where(sq.And{sq.Eq{"rollout_id": rollout.Id}, sq.Eq{"namespace_key": rollout.NamespaceKey}}).
Where(sq.And{sq.Eq{"rollout_id": rollout.Id}}).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Where(sq.And{sq.Eq{"rollout_id": rollout.Id}}).
Where(sq.Eq{"rollout_id": rollout.Id}).


rows, err := builder.Select("segment_key").
From(tableRolloutSegmentReferences).
Where(sq.And{sq.Eq{"rollout_segment_id": rolloutSegmentId}, sq.Eq{"namespace_key": rollout.NamespaceKey}}).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Where(sq.And{sq.Eq{"rollout_segment_id": rolloutSegmentId}, sq.Eq{"namespace_key": rollout.NamespaceKey}}).
Where(sq.Eq{"rollout_segment_id": rolloutSegmentId, "namespace_key": rollout.NamespaceKey}).

Comment on lines 49 to 57
result := make([]string, 0)

if len(segmentKeys) > 0 {
result = append(result, segmentKeys...)
} else if segmentKey != "" {
result = append(result, segmentKey)
}

return removeDuplicates(result)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take it or leave it optimization:

Suggested change
result := make([]string, 0)
if len(segmentKeys) > 0 {
result = append(result, segmentKeys...)
} else if segmentKey != "" {
result = append(result, segmentKey)
}
return removeDuplicates(result)
if len(segmentKeys) > 0 {
return removeDuplicates(segmentKeys)
} else if segmentKey != "" {
return []string{segmentKey}
}
return nil

RunWith(tx).
Set("segment_operator", segmentOperator).
Set("updated_at", &fliptsql.Timestamp{Timestamp: timestamppb.Now()}).
Where(sq.And{sq.Eq{"id": r.Id}, sq.Eq{"namespace_key": r.NamespaceKey}, sq.Eq{"flag_key": r.FlagKey}}).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another small optimization. sq.And and sq.Eq are maps. So each time they're stated we're allocating a map. Squirell with implicity AND all the k/v equalities in a single sq.Eq map.

Suggested change
Where(sq.And{sq.Eq{"id": r.Id}, sq.Eq{"namespace_key": r.NamespaceKey}, sq.Eq{"flag_key": r.FlagKey}}).
Where(sq.Eq{"id": r.Id, "namespace_key": r.NamespaceKey, "flag_key": r.FlagKey}).

@yquansah yquansah merged commit 3f40e63 into main Aug 15, 2023
26 of 27 checks passed
@yquansah yquansah deleted the segment-anding branch August 15, 2023 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants