Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 16.0 GitHub #70

Closed
wants to merge 512 commits into from
Closed

Release 16.0 GitHub #70

wants to merge 512 commits into from

Conversation

alexisjcarr
Copy link

Description

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

harshit-gangal and others added 30 commits January 13, 2023 09:38
GitHub did not seem to pick up the merge conflict between
these two commits (older to newer):
  - vitessio@673573a
  - vitessio@836b3c1

The first commit changed the function signature for
testOnlineDDLStatement(), while the later commit
used the old signature.

Signed-off-by: Matt Lord <[email protected]>

Signed-off-by: Matt Lord <[email protected]>
* Remove dead Workflow mgmt code

This was pre-Legacy sharding, which was in turn pre-VReplication.

Signed-off-by: Matt Lord <[email protected]>

* Remove related proto

Signed-off-by: Matt Lord <[email protected]>

* Get rid of related flag usage

Signed-off-by: Matt Lord <[email protected]>

* Add release notes line and run go mod tidy

Signed-off-by: Matt Lord <[email protected]>

* Add missing command to removed list in release notes

Signed-off-by: Matt Lord <[email protected]>

Signed-off-by: Matt Lord <[email protected]>
* ComBinlogDumpGTID: receive logFile, logPos

Signed-off-by: Shlomi Noach <[email protected]>

* fix error code

Signed-off-by: Shlomi Noach <[email protected]>

* kontinue by default true

Signed-off-by: Shlomi Noach <[email protected]>

* - Added `parseComBinlogDump`
- Added `BinlogEvent.IsHeartbeat()`
- Fixed `NewRotateEvent()` bug
- Added `NewFakeRotateEvent()`
- Added `NewHeartbeatEvent()`
- `mysql/conn.go` handles `ComBinlogDump`
- `mysql/conn.go` handles `ComRegisterReplica`
- Added `Conn.WriteBinlogEvent()`
- `SendBinlogDumpCommand()` accepts a binlog file name, per https://dev.mysql.com/doc/internals/en/com-binlog-dump-gtid.html

Signed-off-by: Shlomi Noach <[email protected]>

* adding a couple unit tests

Signed-off-by: Shlomi Noach <[email protected]>

* updates to go/mysql/binlog_event_make.go

Signed-off-by: Shlomi Noach <[email protected]>

* updates to go/mysql/conn.go

Signed-off-by: Shlomi Noach <[email protected]>

* more tests

Signed-off-by: Shlomi Noach <[email protected]>

* endtoend/cluster: create a cluster without assuming availability of binaries

Signed-off-by: Shlomi Noach <[email protected]>

* endtoend/cluster: more MysqlctlProcess functionality

Signed-off-by: Shlomi Noach <[email protected]>

* indentation

Signed-off-by: Shlomi Noach <[email protected]>

* QueryMatchesTemplates: CanonicalString

Signed-off-by: Shlomi Noach <[email protected]>

* fix comments

Signed-off-by: Shlomi Noach <[email protected]>

Signed-off-by: Shlomi Noach <[email protected]>
* Propagate binlog stream errors

Signed-off-by: Matt Lord <[email protected]>

* Add new unit test

Signed-off-by: Matt Lord <[email protected]>

* Ensure err channel I/O does not block on ctx cancellation

For example, during PRS.

Signed-off-by: Matt Lord <[email protected]>

* Remove unnecessary WaitGroup usage in unit test

Signed-off-by: Matt Lord <[email protected]>

* Log error message when context is closed.

This maintains the previous behavior for the log messages
and ensures that is consistent when the context is closed
or not.

Signed-off-by: Matt Lord <[email protected]>

Signed-off-by: Matt Lord <[email protected]>
* feat: add failing parsing test and fix parser

Signed-off-by: Manan Gupta <[email protected]>

* test: add e2e tests for interval with math functions

Signed-off-by: Manan Gupta <[email protected]>

* test: explictly set the time-zone to prevent failures in CI

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <[email protected]>

* empty-commit

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
…itessio#12098)

* message Rule: convert_int_to_enum

Signed-off-by: Shlomi Noach <[email protected]>

* OnlineDDL/vreplication: handle conversion of integral type to enum type

Signed-off-by: Shlomi Noach <[email protected]>

* admin web proto

Signed-off-by: Shlomi Noach <[email protected]>

* minor naming and comment changes

Signed-off-by: Shlomi Noach <[email protected]>

Signed-off-by: Shlomi Noach <[email protected]>
* store transaction isolation level in upper case

Signed-off-by: Harshit Gangal <[email protected]>

* store system variables in the case as provided in StorageCase type

Signed-off-by: Harshit Gangal <[email protected]>

Signed-off-by: Harshit Gangal <[email protected]>
…12105)

The canonical string representation needs to keep the correct case for
the charset. Those are never upcased in MySQL so we should not enforce
that here either.

Signed-off-by: Dirkjan Bussink <[email protected]>

Signed-off-by: Dirkjan Bussink <[email protected]>
* test: add a failing test for the case where the port changes for a tablet

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix the issue by adding alias as a unique field

Signed-off-by: Manan Gupta <[email protected]>

* empty-commit

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: notfelineit <[email protected]>
* Use Gen4 as the default planner version for VTExplain

Signed-off-by: Florent Poinsard <[email protected]>

* change e2e test output

Signed-off-by: Florent Poinsard <[email protected]>

* use querypb.ExecuteOptions_DEFAULT_PLANNER as the default vtexplain planner version

Signed-off-by: Florent Poinsard <[email protected]>

Signed-off-by: Florent Poinsard <[email protected]>
* notify view change to vtgate

Signed-off-by: Harshit Gangal <[email protected]>

* test: added unit test

Signed-off-by: Harshit Gangal <[email protected]>

* fix: fix e2e test query

Signed-off-by: Harshit Gangal <[email protected]>

Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: notfelineit <[email protected]>
* Update SafeRewrite to allow for replacing a node that we don't want to visit the children of

Signed-off-by: Andres Taylor <[email protected]>

* Clean up ast_rewriting before refactoring it

Signed-off-by: Andres Taylor <[email protected]>

* feat: fix usages of Rewrite in ast_rewriting

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix usages of Rewrite in normalizer

Signed-off-by: Manan Gupta <[email protected]>

* safe rewrite having clause during horizon planning

Signed-off-by: Florent Poinsard <[email protected]>

* move rewriteHavingAndOrderBy to safeRewrite

Signed-off-by: Florent Poinsard <[email protected]>

* refactor: clean up code

Signed-off-by: Andres Taylor <[email protected]>

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>
* feat: remove deprecated fields in logstats

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove enable_semi_sync

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove QueryRowCounts stat

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove deprecated and unused field

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove unused code

Signed-off-by: Manan Gupta <[email protected]>

* release-notes: fix review comments in release-notes

Signed-off-by: Manan Gupta <[email protected]>

* test: remove enable_semi_sync flag from the tests

Signed-off-by: Manan Gupta <[email protected]>

* feat: remove tests for QueryRowCounts

Signed-off-by: Manan Gupta <[email protected]>

* feat: deprecate QueriesProcessed and QueriesRouted

Signed-off-by: Manan Gupta <[email protected]>

* docs: make versions consistant in the release summary

Signed-off-by: Manan Gupta <[email protected]>

Signed-off-by: Manan Gupta <[email protected]>
* Strip secondary keys during schema copy

Signed-off-by: Matt Lord <[email protected]>

* Save ALTER to run when copy finishes

Signed-off-by: Matt Lord <[email protected]>

* Move work to vcopier

Signed-off-by: Matt Lord <[email protected]>

* go mod tidy

Signed-off-by: Matt Lord <[email protected]>

* Re-apply stashed secondary keys

Signed-off-by: Matt Lord <[email protected]>

* Minor fixes and cleanup

Signed-off-by: Matt Lord <[email protected]>

* Add Reshard support

Signed-off-by: Matt Lord <[email protected]>

* Support multiple SQL actions

Signed-off-by: Matt Lord <[email protected]>

* Rename action->>"$.action" to action->>"$.task"

Signed-off-by: Matt Lord <[email protected]>

* Fix unit tests

Signed-off-by: Matt Lord <[email protected]>

* Add new unit test

Signed-off-by: Matt Lord <[email protected]>

* Improve the unit test

Signed-off-by: Matt Lord <[email protected]>

* Add vtctl --defer-secondary-keys client flag

Signed-off-by: Matt Lord <[email protected]>

* Fix binlog unit tests

Signed-off-by: Matt Lord <[email protected]>

* Fix wrangler unit tests

Signed-off-by: Matt Lord <[email protected]>

* Move workflow type check to a function

Signed-off-by: Matt Lord <[email protected]>

* Update error in unit test and add OnlineDDL

Signed-off-by: Matt Lord <[email protected]>

* Minor unit test improvements

Signed-off-by: Matt Lord <[email protected]>

* Improve name of table used

Signed-off-by: Matt Lord <[email protected]>

* Improvements after self review

Signed-off-by: Matt Lord <[email protected]>

* Ensure ALTERs are killed when engine shuts down

Signed-off-by: Matt Lord <[email protected]>

* Fixup & deflake the unit test

Signed-off-by: Matt Lord <[email protected]>

* Minor improvements after self review

Signed-off-by: Matt Lord <[email protected]>

* Auto handle more edge cases

Signed-off-by: Matt Lord <[email protected]>

* More unit test improvements

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Correct new error handling in killAction

Signed-off-by: Matt Lord <[email protected]>

* Improve getTableSecondaryKeys function

Signed-off-by: Matt Lord <[email protected]>

* Nitty comment improvements

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Minor improvements to log message

Signed-off-by: Matt Lord <[email protected]>

* Fix unit tests

Signed-off-by: Matt Lord <[email protected]>

* Add defer_secondary_keys to new unit tests added via main merge

Signed-off-by: Matt Lord <[email protected]>

* More unit test fixes

Signed-off-by: Matt Lord <[email protected]>

* Minor improvements to the log messages

Signed-off-by: Matt Lord <[email protected]>

* Minor changes and a fix after self review

Signed-off-by: Matt Lord <[email protected]>

* Ignore --defer-secondary-keys for Reshard merges

Signed-off-by: Matt Lord <[email protected]>

* Nitty improvement to comment

Signed-off-by: Matt Lord <[email protected]>

* Add full support for shard merges

Signed-off-by: Matt Lord <[email protected]>

* Move newClientConnection() to vreplicator

Signed-off-by: Matt Lord <[email protected]>

* Add unit test for new behavior

Other fixes and minor improvements

Signed-off-by: Matt Lord <[email protected]>

* Use the engine context rather than vreplicator context

Otherwise the ALTER gets KILLed when the copy phase ends

Signed-off-by: Matt Lord <[email protected]>

* Improve testing and other minor tweaks

Signed-off-by: Matt Lord <[email protected]>

* Add post copy action cleanup to workflow cancel

Signed-off-by: Matt Lord <[email protected]>

* More unit test fixes for post copy action cleanup

Signed-off-by: Matt Lord <[email protected]>

Signed-off-by: Matt Lord <[email protected]>
* add debug tooling

Signed-off-by: Andres Taylor <[email protected]>

* move dbg to test

Signed-off-by: Andres Taylor <[email protected]>

Signed-off-by: Andres Taylor <[email protected]>
* more testing for getViewDependentTableNames()

Signed-off-by: Shlomi Noach <[email protected]>

* getForeignKeyParentTableNames

Signed-off-by: Shlomi Noach <[email protected]>

* schema normalization: sort tables by fk reference order

Signed-off-by: Shlomi Noach <[email protected]>

* minor test addition

Signed-off-by: Shlomi Noach <[email protected]>

* ForeignKeyDependencyUnresolvedError

Signed-off-by: Shlomi Noach <[email protected]>

* InvalidReferencedColumnInForeignKeyConstraintError, MismatchingForeignKeyColumnCountError, MismatchingForeignKeyColumnTypeError

Signed-off-by: Shlomi Noach <[email protected]>

* rename error: ForeignKeyColumnCountMismatchError

Signed-off-by: Shlomi Noach <[email protected]>

* rename MismatchingForeignKeyColumnTypeError to ForeignKeyColumnTypeMismatchError

Signed-off-by: Shlomi Noach <[email protected]>

* changes per review

Signed-off-by: Shlomi Noach <[email protected]>

Signed-off-by: Shlomi Noach <[email protected]>
…tld2

[vtctld2] Remove vtctld2 UI and vtctld server components that serve the app UI
GuptaManan100 and others added 21 commits July 28, 2023 17:19
…vitessio#13759)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <[email protected]>
…ob Service (vitessio#13770) (vitessio#13774)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <[email protected]>
…own within CI (vitessio#13793) (vitessio#13797)

Signed-off-by: Matt Lord <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…from schema tracker (vitessio#13673) (vitessio#13795)

Signed-off-by: Matt Lord <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Matt Lord <[email protected]>
…essio#13805)

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>
…ation e2e test (vitessio#13803) (vitessio#13821)

Signed-off-by: Rohit Nayak <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…Diff2 Progress command is in the future. (vitessio#13804) (vitessio#13816)

Signed-off-by: Rohit Nayak <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…uld not be created when a movetables workflow is created (vitessio#13858)

Signed-off-by: Rohit Nayak <[email protected]>
…ules should not be created when a movetables workflow is created" (vitessio#13899)

Signed-off-by: Rohit Nayak <[email protected]>
… migrations (vitessio#13928) (vitessio#13936)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <[email protected]>
@alexisjcarr alexisjcarr self-assigned this Sep 12, 2023
@alexisjcarr alexisjcarr changed the base branch from upstream-release-16.0 to master September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from master to upstream-main September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from upstream-main to upstream-release-16.0 September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from upstream-release-16.0 to master September 12, 2023 22:34
Signed-off-by: Patrick Carnahan <[email protected]>
Signed-off-by: Arthur Schreiber <[email protected]>
@arthurschreiber
Copy link
Member

Closing this in favor of #78, which contains any additional backports we require.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.