Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slack-vitess-r15.0.4: backport v16 vtorc fixes + go1.20 #126

Closed

Conversation

timvaillancourt
Copy link
Member

@timvaillancourt timvaillancourt commented Sep 11, 2023

Description

This PR backports v16 fixes for vtorc -> v15. Also the upgrade to go1.20 was backported (vitessio#12398)

Related PRs

  1. VTOrc running PRS when database_instance empty bug fix. vitessio/vitess#12019
  2. Timeout Fixes and VTOrc Improvement vitessio/vitess#11881
  3. Also log error on a failure in DiscoverInstance vitessio/vitess#11936
  4. VTOrc Code Cleanup - generate_base, replace cluster_name with keyspace and shard. vitessio/vitess#12012
  5. Fix insert query of blocked_recovery table in VTOrc vitessio/vitess#12091
  6. Fix: VTOrc forgetting old instances vitessio/vitess#12089
  7. Upgrade release-16.0 to go1.20.1 vitessio/vitess#12398

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on the CI
  • Documentation was added or is not required

Deployment Notes

* feat: convert join with database_instance to a left join and prevent fixes from running if the information from database_instance is unavailable

Signed-off-by: Manan Gupta <[email protected]>

* test: add tests to verify the fix works

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
* refactor: move tests out of newfeaturestest so that they run on upgrade-downgrade tests too

Signed-off-by: Manan Gupta <[email protected]>

* feat: add failing ers test for handling multiple vttablet failures with default values of flags

Signed-off-by: Manan Gupta <[email protected]>

* feat: add a new lock-timeout flag and use that instead of remote-operation-timeout

Signed-off-by: Manan Gupta <[email protected]>

* feat: augment DownPrimary test to reproduce the issue of VTOrc not handling multiple failures

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove LockShardTimeout configuration from VTOrc and add parallelism to refresh of tablets

Signed-off-by: Manan Gupta <[email protected]>

* log: add more logging lines around ers in vtorc

Signed-off-by: Manan Gupta <[email protected]>

* test: get the test to work

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix usage of wait for replicas timeout

Signed-off-by: Manan Gupta <[email protected]>

* test: fix flags expected output

Signed-off-by: Manan Gupta <[email protected]>

* test: fix race in test now that the function is called in parallel multiple times

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix default of onCloseTimeout to 1 second

Signed-off-by: Manan Gupta <[email protected]>

* test: add failing unit test to refreshTabletsInKeyspaceShard

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix vtorc to not forget a tablet which has been deleted

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix backward compatibility, add tests and release notes docs

Signed-off-by: Manan Gupta <[email protected]>

* test: fix flags output

Signed-off-by: Manan Gupta <[email protected]>

* test: use disable-replication-manager instead of disable-active-reparents to allow vttablets to setup replication when restarted

Signed-off-by: Manan Gupta <[email protected]>

* test: fix flaky test by not checking for an error

Signed-off-by: Manan Gupta <[email protected]>

* feat: handle the case of empty hostname in tablet initialization

Signed-off-by: Manan Gupta <[email protected]>

* feat: update onclose timeout to 10 seconds

Signed-off-by: Manan Gupta <[email protected]>

* test: fix unit test

Signed-off-by: Manan Gupta <[email protected]>

* feat: address review comments

Signed-off-by: Manan Gupta <[email protected]>

* docs: add comments explaining the test functions

Signed-off-by: Manan Gupta <[email protected]>

* feat: add summary docs for 'lock-shard-timeout' deprecation

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
…e and shard. (vitessio#12012)

* feat: refactor generate commands of VTOrc to be in a single file

Signed-off-by: Manan Gupta <[email protected]>

* refactor: cleanup create table formatting

Signed-off-by: Manan Gupta <[email protected]>

* feat: cleanup the usage of IsSQLite and IsMySQL

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove unused minimal instance

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove unused table cluster_domain_name

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix vtorc database to store keyspace and shard instead of cluster

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove unused attributes

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove unused cluster domain

Signed-off-by: Manan Gupta <[email protected]>

* feat: change GetClusterName to GetKeyspaceAndShardName

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix insertion into database_instance

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix SnapshotTopologies

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove inject unseen primary and inject seed

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove ClusterName from Instance

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix Audit operations

Signed-off-by: Manan Gupta <[email protected]>

* feat: add Keyspace and Shard to cluster information to replace ClusterName

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix attempt failure detection registeration

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix blocked topology recoveries

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix topology recovery

Signed-off-by: Manan Gupta <[email protected]>

* feat: reading recovery instances

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix get replication and analysis

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix bug in query

Signed-off-by: Manan Gupta <[email protected]>

* test: add tests to check that filtering by keyspace works for APIs

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove remaining usages of ClusterName

Signed-off-by: Manan Gupta <[email protected]>

* refactor: fix comment explaining sleep in the test

Signed-off-by: Manan Gupta <[email protected]>

* feat: add code to prevent filtering just by shard and add tests for it

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <[email protected]>

* empty-commit

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
* test: add a failing test for the case where the port changes for a tablet

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix the issue by adding alias as a unique field

Signed-off-by: Manan Gupta <[email protected]>

* empty-commit

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
@salesforce-cla
Copy link

Thanks for the contribution! Before we can merge this, we need @GuptaManan100 to sign the Salesforce Inc. Contributor License Agreement.

timvaillancourt and others added 2 commits September 12, 2023 17:47
* Upgrade release-16.0 to go1.19.6

Signed-off-by: Florent Poinsard <[email protected]>

* upgrade to 1.20.1

Signed-off-by: Florent Poinsard <[email protected]>

* bump the golangci-lint version

Signed-off-by: Florent Poinsard <[email protected]>

* Apply linter suggestions

Signed-off-by: Florent Poinsard <[email protected]>

* Apply linter suggestions

Signed-off-by: Florent Poinsard <[email protected]>

* fix TestTLSClientVerifyIdentity

Signed-off-by: Florent Poinsard <[email protected]>

* Switch to using new Go 1.19 CRL parser (vitessio#12315)

`x509.ParseCRL` is deprecated, we should use `x509.ParseRevocationList`
instead which is new in Go 1.19.

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>

* Skip TestFuzz for now

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Dirkjan Bussink <[email protected]>
Co-authored-by: Dirkjan Bussink <[email protected]>
@salesforce-cla
Copy link

Thanks for the contribution! Before we can merge this, we need @GuptaManan100 @frouioui to sign the Salesforce Inc. Contributor License Agreement.

@timvaillancourt timvaillancourt changed the title slack-vitess-r15.0.4: backport v16 vtorc fixes slack-vitess-r15.0.4: backport v16 vtorc fixes + go1.20 Sep 12, 2023
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
@timvaillancourt timvaillancourt deleted the bp-v16-vtorc-slack-vitess-r15.0.4 branch October 9, 2023 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants