Timeplus Support

[jovezhong]

Timeplus is a simple, powerful, and cost-efficient stream processing platform. This PR added support for both Timeplus Proton (https://github.com/timeplus-io/proton) and Timeplus Enterprise (https://timeplus.com/product). The plugin is tested with Timeplus Enterprise and verified with one of our customers who also purchased flyway.

To test it:

Start a docker instance of Timeplus Enterprise via

docker run --name tpe2.5.11 -p 8000:8000 -p 8463:8463  -p 7587:7587 timeplus/timeplus-enterprise:2.5.11

Then you can access http://localhost:8000 and create the first user, say admin, with password changeme.

In the working folder, create flyway.toml

[environments.tplocal]
url = "jdbc:timeplus://localhost:7587"
user = "admin"
password = "changeme"
[flyway]
validateMigrationNaming = true
environment = "tplocal"
locations = ["filesystem:migrations"]
cleanDisabled = false

and create some SQL files in the migrations folder.

Download the JDBC jar from https://github.com/timeplus-io/timeplus-native-jdbc/releases/tag/v2.0.7 and put in lib folder.

---

PS. the code is largely based on the ClickHouse database support, with necessary SQL keywords or data type names changes.

[jovezhong]

I put a 3-min demo video on the doc page https://docs.timeplus.com/flyway

[WilliamL-RG]

In TimeplusDatabase, there is no override for the getUpdateStatement which is called by Flyway as part of running Repair. I have tried testing this and ran into an error due to the difference syntax with Timeplus and updating.
Is this a known issue and a limitation or something that needs to be implemented?

[jovezhong]

Hi @WilliamL-RG , thanks for looking into this. Timeplus is similar to ClickHouse, basically is a OLAP focusing on append-only data. I don't see clickhouse flyway extension implemented the getUpdateStatement. The base.Database will generate UPDATE <table> SET .. which won't work in Timeplus.

In Timeplus Enterprise edition, we built a row-based storage using RocksDB timeplus.com/post/introducing-mutable-streams. In this stream, the update is done by appending a row with same primary key(s)

For the flyway metadata table, in this PR, I created it with a ClickHouse MergeTree engine, which should support lightweight update
https://clickhouse.com/docs/en/sql-reference/statements/alter/update
using ALTER TABLE [db.]table [ON CLUSTER cluster] UPDATE column1 = expr1 [, ...] [IN PARTITION partition_id] WHERE filter_expr

Please advise whether flyway repair will update the metadata table or the actual data table/streams. I think I tried flyway repair and it can correct metadata rows while failed execution.

Open for any suggestions

[jovezhong]

Hi @WilliamL-RG , I added a commit to override the getUpdateStatement method, please review and advise.