Merge pull request #8 from tarantool/providers/etcdv3

Add viper remote etcd v3 config tests & update golang version to 1.22

Author: KaymeKaydex

.github/workflows/main.yml

@@ -19,15 +19,15 @@ jobs:
       fail-fast: false
       matrix:
         golang:
-          - '1.20'
+          - '1.22'
           - 'stable'
         tarantool:
           - '2.8'
           - '2.10'
           - 'master'
         include:
           - tarantool: 'master'
-            golang: '1.20'
+            golang: '1.22'
             coverage: true
 
     steps:

CHANGELOG.md

@@ -1,3 +1,17 @@
+## Unreleased
+
+CHANGES:
+
+* Updated Go version from 1.20 to 1.22.
+
+TESTS:
+
+* Added a test for using etcd v3 with Viper.
+
+REFACTOR:
+
+* Added information about available providers to the README.
+
 ## v1.3.1
 
 REFACTOR:
@@ -16,15 +30,18 @@ EXAMPLES:
 ## v1.3.0
 
 BUG FIXES:
+
 * Fix decoding fields for StorageCallVShardError (MasterUUID, ReplicasetUUID).
 
 CHANGES:
+
 * Add comment why and how we handle "NON_MASTER" vshard error.
 * Don't support 'type Error struct' anymore.
 * Linter: don't capitalize error strings and capitalize log.
 * Fix misspellings.
 * Handle vshard error the same way as lua vshard router (resolve issue #77).
-* Decode 'vshard.storage.call' response manually into struct vshardStorageCallResponseProto using DecodeMsgpack interface to reduce allocations (partially #61, #100).
+* Decode 'vshard.storage.call' response manually into struct vshardStorageCallResponseProto using DecodeMsgpack
+  interface to reduce allocations (partially #61, #100).
 * Remove `mapstructure` tag from StorageCallVShardError.
 * Update benchmarks in README files.
 * Package `mapstructure` is completely removed from direct dependencies list.
@@ -48,15 +65,18 @@ REFACTOR:
 * New backward-compatible signature for StorageResultTypedFunc to fix interface for RouterCallImpl.
 
 TESTS:
+
 * Rename bootstrap_test.go -> tarantool_test.go and new test in this file.
 * Test for new 'RouterMapCallRW[T]'.
 
 ## v1.2.0
 
 CHANGES:
+
 * We don't support LogProvider interface anymore, only LogfProvider should be used.
 
 BUG FIXES:
+
 * RouterCallImpl: retry on BucketResolve error.
 * RouterCallImpl: do not retry on vshard error "TRANSFER_IS_IN_PROGRESS".
 * RouterCallImpl: remove misleading RetryOnCall.
@@ -77,17 +97,21 @@ REFACTOR:
 * Support bucketSearchBatched method for batched buckets discovery (resolve #71).
 
 TESTS:
+
 * Tests for BucketsSearchMode (tnt/discovery_test.go).
-* The tests have been rewritten in such a way that now you can configure the cluster to suit any needs, the configuration of the clusters may differ and the launch is controlled from go.
+* The tests have been rewritten in such a way that now you can configure the cluster to suit any needs, the
+  configuration of the clusters may differ and the launch is controlled from go.
 * Legacy integration tests removed (tests/integration).
 
 ## v1.1.0
 
 CHANGES:
+
 * now if there is no etcd connection - etcd provider returns you error instead panic.
 * etcd provider also accepts context as input for working with the logger.
 
 BUG FIXES:
+
 * fixed the problem of re-creating a new context when starting discovery, which led to a lack of discovery logging.
 
 TESTS:
@@ -96,13 +120,15 @@ TESTS:
 * Indent fix for test/tnt/storage.lua.
 
 FEATURES:
+
 * now we write topology changes to debug logs.
 * now we write info logs about new replicaset adding and nodes state.
 * added pull request template to repository.
 
 ## v1.0.1
 
 BUG FIXES:
+
 * fix invalid go-tarantool backward compatibility: NewCallRequest not method of router.
 
 ## 1.0.0
@@ -123,7 +149,7 @@ FEATURES:
 * DiscoveryTimeout by default is 1 minute (zero DiscoveryTimeout is not allowed #60)
 * All discovering logs has new prefix [DISCOVERY]
 * Introduce Replicaset.CallAsync, it is usefull to send concurrent requests to replicasets;
-	additionally, CallAsync provides new interface to interact with replicaset without cons of interface of ReplicaCall
+  additionally, CallAsync provides new interface to interact with replicaset without cons of interface of ReplicaCall
 * Retry tarantool request only on some vshard errors (#66).
 * Added call interfaces backwards compatible with go-tarantool (#31).
 
@@ -146,15 +172,16 @@ TESTS:
 * New tnt tests for RouterMapCallRWImpl
 * New tnt tests for topology logic
 * Big CI update
-  * 2 sections for CI: static checks and tests
-  * integration tests run on ci with Tarantool cluster on vshard
-  * implemented luacheck for static checks
+    * 2 sections for CI: static checks and tests
+    * integration tests run on ci with Tarantool cluster on vshard
+    * implemented luacheck for static checks
 * New tnt tests for ReplicaCall
 * New tnt tests for CallAsync
 * Init Go benches for tests/tnt
 
 EXAMPLES:
-* customer go mod fixed 
+
+* customer go mod fixed
 * add customer example listen addr log
 
 ## 0.0.12
@@ -191,7 +218,6 @@ REFACTOR:
 * Makefile refactored for racetests
 * Tests coverage up 22% -> 33%
 
-
 ## 0.0.11
 
 BUG FIXES:

README.md

@@ -182,6 +182,17 @@ func main() {
   fmt.Printf("get typed result: %v", info)
 }
 ```
+### Providers
+You can use topology (configuration) providers as the source of router configuration.  
+Currently, the following providers are supported:
+
+- **etcd** (for configurations similar to [moonlibs/config](https://github.com/moonlibs/config?tab=readme-ov-file#example-of-etcd-configuration-for-vshard-based-applications-etcdclustervshard) in etcd v2 for Tarantool versions below 3)
+- **static** (for specifying configuration directly in the code for ease of testing)
+- **[viper](providers/viper/README.md)**
+  - etcd v3
+  - consul
+  - files
+
 ### Learn more examples
 #### Quick Start
 Learn with th [Quick Start](docs/doc.md), which include  examples and theory.

README_ru.md

@@ -182,6 +182,17 @@ func main() {
 }
 
 ```
+
+### Провайдеры
+Как источник конфигурации вы можете использовать провайдеры топологии(конфигурации).
+На данный момент есть поддержка следующих провайдеров:
+- **etcd** (для конфигурации аналогичной [moonlibs/config](https://github.com/moonlibs/config?tab=readme-ov-file#example-of-etcd-configuration-for-vshard-based-applications-etcdclustervshard) в etcd v2 для tarantool версии ниже 3)
+- **static** (для указания конфигурации прямо из кода и простоты тестирования)
+- **[viper](providers/viper/README.md)**
+  - etcd v3
+  - consul
+  - files
+
 ### Ознакомьтесь с другими примерами
 #### Быстрое начало
 Познакомьтесь с  [Полной документацией](docs/doc_ru.md), которая включает в себя примеры и теорию.

examples/customer/go-service/go.mod

@@ -3,11 +3,11 @@ module customer
 go 1.22.2
 
 require (
-	github.com/tarantool/go-vshard-router v0.0.9
 	github.com/google/uuid v1.6.0
-	github.com/spf13/viper v1.18.2
+	github.com/spf13/viper v1.19.0
 	github.com/swaggo/swag v1.16.3
 	github.com/tarantool/go-tarantool/v2 v2.1.1-0.20240922082035-592db69eed86
+	github.com/tarantool/go-vshard-router v0.0.9
 )
 
 require (
@@ -24,7 +24,7 @@ require (
 	github.com/magiconair/properties v1.8.7 // indirect
 	github.com/mailru/easyjson v0.7.6 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
-	github.com/pelletier/go-toml/v2 v2.1.0 // indirect
+	github.com/pelletier/go-toml/v2 v2.2.2 // indirect
 	github.com/sagikazarmark/locafero v0.4.0 // indirect
 	github.com/sagikazarmark/slog-shim v0.1.0 // indirect
 	github.com/snksoft/crc v1.1.0 // indirect
@@ -39,9 +39,9 @@ require (
 	go.uber.org/atomic v1.9.0 // indirect
 	go.uber.org/multierr v1.9.0 // indirect
 	golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
-	golang.org/x/net v0.19.0 // indirect
+	golang.org/x/net v0.23.0 // indirect
 	golang.org/x/sync v0.6.0 // indirect
-	golang.org/x/sys v0.15.0 // indirect
+	golang.org/x/sys v0.18.0 // indirect
 	golang.org/x/text v0.14.0 // indirect
 	golang.org/x/tools v0.13.0 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect

examples/customer/go-service/go.sum

@@ -50,6 +50,7 @@ github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWb
 github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
 github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
 github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
+github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -71,6 +72,7 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/viper v1.18.2 h1:LUXCnvUvSM6FXAsj6nnfc8Q2tp1dIgUfY9Kc8GsSOiQ=
 github.com/spf13/viper v1.18.2/go.mod h1:EKmWIqdnk5lOcmR72yw6hS+8OPYcwD0jteitLMVB+yk=
+github.com/spf13/viper v1.19.0/go.mod h1:GQUN9bilAbhU/jgc1bKs99f/suXKeUMct8Adx5+Ntkg=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -107,12 +109,14 @@ golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
 golang.org/x/net v0.0.0-20210421230115-4e50805a0758/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM=
 golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
 golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
+golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
 golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
 golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210420072515-93ed5bcd2bfe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
 golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=

go.mod

@@ -1,47 +1,135 @@
 module github.com/tarantool/go-vshard-router
 
-go 1.20
+go 1.22
+
+toolchain go1.23.3
 
 require (
 	github.com/google/uuid v1.6.0
 	github.com/snksoft/crc v1.1.0
-	github.com/spf13/viper v1.18.2
+	github.com/spf13/viper v1.19.0
 	github.com/stretchr/testify v1.9.0
 	github.com/tarantool/go-tarantool/v2 v2.1.1-0.20240922082035-592db69eed86
 	github.com/vmihailenco/msgpack/v5 v5.3.5
-	go.etcd.io/etcd/client/v2 v2.305.10
+	go.etcd.io/etcd/client/v2 v2.305.17
+	go.etcd.io/etcd/client/v3 v3.5.17
+	go.etcd.io/etcd/server/v3 v3.5.17
 	golang.org/x/sync v0.6.0
 )
 
 require (
+	cloud.google.com/go v0.112.1 // indirect
+	cloud.google.com/go/compute v1.24.0 // indirect
+	cloud.google.com/go/compute/metadata v0.2.3 // indirect
+	cloud.google.com/go/firestore v1.15.0 // indirect
+	cloud.google.com/go/longrunning v0.5.5 // indirect
+	github.com/armon/go-metrics v0.4.1 // indirect
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cenkalti/backoff/v4 v4.2.1 // indirect
+	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/coreos/go-semver v0.3.0 // indirect
+	github.com/coreos/go-systemd/v22 v22.3.2 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
+	github.com/dustin/go-humanize v1.0.0 // indirect
+	github.com/fatih/color v1.14.1 // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
+	github.com/go-logr/logr v1.4.1 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
+	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
+	github.com/google/btree v1.0.1 // indirect
+	github.com/google/s2a-go v0.1.7 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
+	github.com/googleapis/gax-go/v2 v2.12.3 // indirect
+	github.com/gorilla/websocket v1.4.2 // indirect
+	github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
+	github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
+	github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
+	github.com/hashicorp/consul/api v1.28.2 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
+	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
+	github.com/hashicorp/go-hclog v1.5.0 // indirect
+	github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
+	github.com/hashicorp/go-multierror v1.1.1 // indirect
+	github.com/hashicorp/go-rootcerts v1.0.2 // indirect
+	github.com/hashicorp/golang-lru v0.5.4 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
+	github.com/hashicorp/serf v0.10.1 // indirect
+	github.com/jonboulle/clockwork v0.2.2 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/klauspost/compress v1.17.2 // indirect
 	github.com/magiconair/properties v1.8.7 // indirect
+	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/mattn/go-isatty v0.0.17 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
+	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
-	github.com/pelletier/go-toml/v2 v2.1.0 // indirect
+	github.com/nats-io/nats.go v1.34.0 // indirect
+	github.com/nats-io/nkeys v0.4.7 // indirect
+	github.com/nats-io/nuid v1.0.1 // indirect
+	github.com/pelletier/go-toml/v2 v2.2.2 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
+	github.com/prometheus/client_golang v1.11.1 // indirect
+	github.com/prometheus/client_model v0.2.0 // indirect
+	github.com/prometheus/common v0.26.0 // indirect
+	github.com/prometheus/procfs v0.6.0 // indirect
+	github.com/sagikazarmark/crypt v0.19.0 // indirect
 	github.com/sagikazarmark/locafero v0.4.0 // indirect
 	github.com/sagikazarmark/slog-shim v0.1.0 // indirect
+	github.com/sirupsen/logrus v1.9.3 // indirect
+	github.com/soheilhy/cmux v0.1.5 // indirect
 	github.com/sourcegraph/conc v0.3.0 // indirect
 	github.com/spf13/afero v1.11.0 // indirect
 	github.com/spf13/cast v1.6.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/stretchr/objx v0.5.2 // indirect
 	github.com/subosito/gotenv v1.6.0 // indirect
 	github.com/tarantool/go-iproto v1.0.0 // indirect
+	github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
 	github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
-	go.etcd.io/etcd/api/v3 v3.5.10 // indirect
-	go.etcd.io/etcd/client/pkg/v3 v3.5.10 // indirect
+	github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
+	go.etcd.io/bbolt v1.3.11 // indirect
+	go.etcd.io/etcd/api/v3 v3.5.17 // indirect
+	go.etcd.io/etcd/client/pkg/v3 v3.5.17 // indirect
+	go.etcd.io/etcd/pkg/v3 v3.5.17 // indirect
+	go.etcd.io/etcd/raft/v3 v3.5.17 // indirect
+	go.opencensus.io v0.24.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
+	go.opentelemetry.io/otel v1.24.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.20.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.20.0 // indirect
+	go.opentelemetry.io/otel/metric v1.24.0 // indirect
+	go.opentelemetry.io/otel/sdk v1.22.0 // indirect
+	go.opentelemetry.io/otel/trace v1.24.0 // indirect
+	go.opentelemetry.io/proto/otlp v1.0.0 // indirect
 	go.uber.org/atomic v1.9.0 // indirect
 	go.uber.org/multierr v1.9.0 // indirect
+	go.uber.org/zap v1.21.0 // indirect
+	golang.org/x/crypto v0.21.0 // indirect
 	golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
-	golang.org/x/sys v0.15.0 // indirect
+	golang.org/x/net v0.23.0 // indirect
+	golang.org/x/oauth2 v0.18.0 // indirect
+	golang.org/x/sys v0.18.0 // indirect
 	golang.org/x/text v0.14.0 // indirect
+	golang.org/x/time v0.5.0 // indirect
+	google.golang.org/api v0.171.0 // indirect
+	google.golang.org/appengine v1.6.8 // indirect
+	google.golang.org/genproto v0.0.0-20240213162025-012b6fc9bca9 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20240311132316-a219d84964c2 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20240314234333-6e1732d8331c // indirect
+	google.golang.org/grpc v1.62.1 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
+	gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
+	sigs.k8s.io/yaml v1.2.0 // indirect
 )