Clusterpedia v0.8.1

What's Changed

• [release/0.8] fix the version order of 'discovery.k8s.io' by @Iceber in #660

Full Changelog: v0.8.0...v0.8.1

Clusterpedia v0.7.2

What's Changed

• [release/0.7] fix the version order of 'discovery.k8s.io' by @Iceber in #661

Full Changelog: v0.7.1...v0.7.2

Clusterpedia v0.7.2-rc.0

What's Changed

• [release/0.7] fix the version order of 'discovery.k8s.io' by @Iceber in #661

Full Changelog: v0.7.1...v0.7.2-rc.0

Clusterpedia v0.8.0

Notable Changes since v0.7.0

APIService

• add clusterpedia openapi (@wawa0210 @huiwq1990, #579, #588)

The kube apiserver no longer prints the following error log on a recurring basis:

E1225 07:23:43.660066       1 controller.go:116] loading OpenAPI spec for "v1beta1.clusterpedia.io" failed with: OpenAPI spec does not exist
I1225 07:23:43.660079       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.clusterpedia.io: Rate Limited Requeue.

• add OTEL trace config for clusterpedia-apiserver (@KubeKyrie, #604)

Add the --tracing-config-file flag to allow configuration of Otel trace, example configuration

apiVersion: apiserver.config.k8s.io/v1beta1
kind: TracingConfiguration
# default
# endpoint: localhost:4317
samplingRatePerMillion: 1000000

Detailed configuration can be found in TracingConfiguration

ClusterSynchrManager

• handle each of the pages in the resource list stage (@Iceber, #591)

This feature will reduce the maximum memory footprint for resource synchronisation.

It is controlled by two feature gates:

• StreamHandlePaginatedListForResourceSync: stream handling of paged resources, resources within a pager will be processed as soon as possible instead of waiting until all resources are pulled before calling the ResourceHandler.
• ForcePaginatedListForResourceSync: force paging, Reflector will sometimes use APIServer's cache, even if paging is specified APIServer will return all resources for performance, then it will skip Reflector's streaming memory optimization.

ForcePaginatedListForResourceSync may put extra pressure on the imported cluster APIServer, please be careful to enable it.

clustersynchro-manager add --page-size flags, you can control the page size of the List, the smaller the page size, the smaller the memory consumption, but may also generate more requests to APIServer.

In the future we may be able to configure these in PediaCluster.

---

• add sharding feature for ClusterSynchroManager (@jxustc, #609)
  In order to avoid a single resource synchroniser taking up a lot of memory, it is now possible to synchronise the PediaCluster in shards.

In two steps:

1. Start a new ClusterSynchroManager Deploymnet or Pod and set the sharding name with --sharding-name flag.
2. Configure the spec.shardingName field of PediaCluster

PediaCluster will only be synchronised by ClusterSynchroManagers with the same sharding name.

---

• add a feature gate IgnoreSyncLease to stop sync leases.coordination.k8s.io (@27149chen, #615, #622)

The leases resource usually needs to be skipped when using resource wildcards or synchronising full resources, and this feature gating ensures that the leases resource is globally skipped without modifying PediaCluster.

In the future, it is more recommended to use #617 to exclude certain resource types.

Multi-Cluster Resource State Metrics

• Support resource state metrics like kube-state-metrics (@Iceber, #539)
• Support for exposing single cluster metrics (@Iceber, #553)
• add kube state metrics namespace command (@scydas, #566)

The cluster synchro has the same source data as kube-state-metrics available to it at sync time, so clusterpedia can easily support functionality similar to kube-state-metrics

# HELP kube_deployment_created [STABLE] Unix creation timestamp
# TYPE kube_deployment_created gauge
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="kube-system",deployment="coredns"} 1.673603507e+09
# HELP kube_deployment_status_replicas [STABLE] The number of replicas per deployment.
# TYPE kube_deployment_status_replicas gauge
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 0
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="kube-system",deployment="coredns"} 2

Added --enable-kube-state-metrics and a set of related flags to clustersynchro-manager to control multi-cluster kube-state-metrics capability.

$ ./bin/clustersynchro-manager -h
Metrics server flags:
    ...
Kube state metrics flags:
    ...

Storage

• convert deleted objects according to the ConvertDeletedObject method of the storage layer (@huiwq1990 @Iceber, #592, #625)

InternalStorage

• make DB bad connection error message more specific (@KubeKyrie, #598)

Returns a more readable error message(storage error: database connection error: bad connection) in the case of driver.DBError

• Added new parameters for parameterized query (@nekomeowww, #597)

Existing url query - whereSQL needs to splice the value directly in sql, which can easily lead to sql injection problems.

We added three url queries to support parameterized query，give users the ability to prevent sql injection.

url query	desc	example
whereSQLStatement	Splice the sql statement after WHERE,Use ? to represent the value.	whereSQLStatement=(cluster=?) AND (namespace=(?))
whereSQLParam	Multiple whereSQLParam can be set, each corresponding to a question mark.	whereSQLParam=docker-desktop&whereSQLParam=default
whereSQLJSONParams	whereSQLParam has the obvious drawback that it can't pass complex structures and special types such as arrays, etc., so you can use whereSQLJSONParam to do so.	whereSQLStatement=(cluster=?) AND (namespace IN (?))&whereSQLJSONParams=WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K

whereSQLJSONParam is a base64 of an array:

$  echo WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K | base64 -d
["docker-desktop",["default","kube-system"]]

Parameterised SQL queries are enabled by the AllowParameterizedSQLQuery feature gate

BUG FIX

• fix: direct []byte write when updating resources (@nekomeowww, #607)
• update the created_at field when updating resource (@flpanbin, #629)

Others

• Bump golang from 1.19.5 to 1.21.4 (@dependabot @khareyash05, #537, #552, #581, #587, #610)

• Bump alpine from 3.17.3 to 3.18.4 (@dependabot, #549, #554, #573)

• Bump Kubernetes from v1.26.3 to v1.28.4 (@Iceber @wawa0210, #538, #579, #606)

• Use kustomize to install clusterpedia (@khareyash05 @Iceber, #551, #556, #558)

$ kubectl kustomize ./kustomize | kubectl apply -f -

• Use kustomize for e2e testing and remove the helm chart (@khareyash05, #560, #563)

Contributers

Thanks everyone who contributed to this release!

The following users (sort alphabetically) are those who committed much in this release. Thank you!

• @27149chen
• @flpanbin
• @hexiaodai
• @huiwq1990
• @Iceber
• @jxustc
• @khareyash05
• @KubeKyrie
• @mengjiao-liu
• @nekomeowww
• @rokkiter
• @scydas
• @wawa0210
• @wzshiming

Clusterpedia v0.8.0-beta.0

Notable Changes since v0.7.0

APIService

• add clusterpedia openapi (@wawa0210 @huiwq1990, #579, #588)

The kube apiserver no longer prints the following error log on a recurring basis:

E1225 07:23:43.660066       1 controller.go:116] loading OpenAPI spec for "v1beta1.clusterpedia.io" failed with: OpenAPI spec does not exist
I1225 07:23:43.660079       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.clusterpedia.io: Rate Limited Requeue.

• add OTEL trace config for clusterpedia-apiserver (@KubeKyrie, #604)

Add the --tracing-config-file flag to allow configuration of Otel trace, example configuration

apiVersion: apiserver.config.k8s.io/v1beta1
kind: TracingConfiguration
# default
# endpoint: localhost:4317
samplingRatePerMillion: 1000000

Detailed configuration can be found in TracingConfiguration

ClusterSynchrManager

• handle each of the pages in the resource list stage (@Iceber, #591)

This feature will reduce the maximum memory footprint for resource synchronisation.

It is controlled by two feature gates:

• StreamHandlePaginatedListForResourceSync: stream handling of paged resources, resources within a pager will be processed as soon as possible instead of waiting until all resources are pulled before calling the ResourceHandler.
• ForcePaginatedListForResourceSync: force paging, Reflector will sometimes use APIServer's cache, even if paging is specified APIServer will return all resources for performance, then it will skip Reflector's streaming memory optimization.

ForcePaginatedListForResourceSync may put extra pressure on the imported cluster APIServer, please be careful to enable it.

clustersynchro-manager add --page-size flags, you can control the page size of the List, the smaller the page size, the smaller the memory consumption, but may also generate more requests to APIServer.

In the future we may be able to configure these in PediaCluster.

---

• add sharding feature for ClusterSynchroManager (@jxustc, #609)
  In order to avoid a single resource synchroniser taking up a lot of memory, it is now possible to synchronise the PediaCluster in shards.

In two steps:

1. Start a new ClusterSynchroManager Deploymnet or Pod and set the sharding name with --sharding-name flag.
2. Configure the spec.shardingName field of PediaCluster

PediaCluster will only be synchronised by ClusterSynchroManagers with the same sharding name.

---

• add a feature gate IgnoreSyncLease to stop sync leases.coordination.k8s.io (@27149chen, #615, #622)

The leases resource usually needs to be skipped when using resource wildcards or synchronising full resources, and this feature gating ensures that the leases resource is globally skipped without modifying PediaCluster.

In the future, it is more recommended to use #617 to exclude certain resource types.

Multi-Cluster Resource State Metrics

• Support resource state metrics like kube-state-metrics (@Iceber, #539)
• Support for exposing single cluster metrics (@Iceber, #553)
• add kube state metrics namespace command (@scydas, #566)

The cluster synchro has the same source data as kube-state-metrics available to it at sync time, so clusterpedia can easily support functionality similar to kube-state-metrics

# HELP kube_deployment_created [STABLE] Unix creation timestamp
# TYPE kube_deployment_created gauge
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1.676557618e+09
kube_deployment_created{cluster="test-14",namespace="kube-system",deployment="coredns"} 1.673603507e+09
# HELP kube_deployment_status_replicas [STABLE] The number of replicas per deployment.
# TYPE kube_deployment_status_replicas gauge
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-apiserver"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-clustersynchro-manager"} 0
kube_deployment_status_replicas{cluster="test-14",namespace="clusterpedia-system",deployment="clusterpedia-controller-manager"} 1
kube_deployment_status_replicas{cluster="test-14",namespace="kube-system",deployment="coredns"} 2

Added --enable-kube-state-metrics and a set of related commands, flags, to clustersynchro-manager to control multi-cluster kube-state-metrics capability.

$ ./bin/clustersynchro-manager -h
Metrics server flags:

      --metrics-disable-gzip-encoding
                Gzip responses when requested by clients via 'Accept-Encoding: gzip' header.
      --metrics-host string
                Host to expose clustersynchro manager metrics on. (default "::")
      --metrics-port int
                Port to expose clustersynchro manager metrics on. (default 8081)
      --metrics-tls-config string
                Path to the TLS configuration file of metrics

Kube state metrics flags:

      --enable-kube-state-metrics
                Enabled kube state metrics
      --kube-state-metrics-host string
                Host to expose kube state metrics on. (default "::")
      --kube-state-metrics-metric-allowlist string
                Comma-separated list of metrics to be exposed. This list comprises of exact metric names
                and/or regex patterns. The allowlist and denylist are mutually exclusive.
      --kube-state-metrics-metric-denylist string
                Comma-separated list of metrics not to be enabled. This list comprises of exact metric
                names and/or regex patterns. The allowlist and denylist are mutually exclusive.
      --kube-state-metrics-metric-opt-in-list string
                Comma-separated list of metrics which are opt-in and not enabled by default. This is in
                addition to the metric allow- and denylists
      --kube-state-metrics-namespaces string
                Comma-separated list of namespaces to be enabled. Defaults to ""
      --kube-state-metrics-namespaces-denylist string
                Comma-separated list of namespaces not to be enabled. If namespaces and
                namespaces-denylist are both set, only namespaces that are excluded in namespaces-denylist
                will be used.
      --kube-state-metrics-port int
                Port to expose kube state metrics on. (default 8080)
      --kube-state-metrics-resources string
                Comma-separated list of Resources to be enabled. Supported resources:
                "cronjobs,daemonsets,deployments,ingressclasses,ingresses,jobs,namespaces,nodes,pods,replicasets,secrets,services,statefulsets" (default "deployments,pods,services")

Storage

• convert deleted objects according to the ConvertDeletedObject method of the storage layer (@huiwq1990 @Iceber, #592, #625)

InternalStorage

• make DB bad connection error message more specific (@KubeKyrie, #598)

Returns a more readable error message(storage error: database connection error: bad connection) in the case of driver.DBError

• Added new parameters for parameterized query (@nekomeowww, #597)

Existing url query - whereSQL needs to splice the value directly in sql, which can easily lead to sql injection problems.

We added three url queries to support parameterized query，give users the ability to prevent sql injection.

url query	desc	example
whereSQLStatement	Splice the sql statement after WHERE,Use ? to represent the value.	whereSQLStatement=(cluster=?) AND (namespace=(?))
whereSQLParam	Multiple whereSQLParam can be set, each corresponding to a question mark.	whereSQLParam=docker-desktop&whereSQLParam=default
whereSQLJSONParam	whereSQLParam has the obvious drawback that it can't pass complex structures and special types such as arrays, etc., so you can use whereSQLJSONParam to do so.	whereSQLStatement=(cluster=?) AND (namespace IN (?))&whereSQLJSONParams=WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K

whereSQLJSONParam is a base64 of an array:

$  echo WyJkb2NrZXItZGVza3RvcCIsWyJkZWZhdWx0Iiwia3ViZS1zeXN0ZW0iXV0K | base64 -d
["docker-desktop",["default","kube-system"]]

Parameterised SQL queries are enabled by the AllowParameterizedSQLQuery feature gate

BUG FIX

• fix: direct []byte write when updating resources (@nekomeowww, #607)
• update the created_at field when updating resource (@flpanbin, #629)

Others

• Bump golang fr...

Clusterpedia v0.8.0-alpha.1

Merge pull request #625 from Iceber/convert_deleted_object

add ConvertDeletedObject method for ResourceStorage

Clusterpedia v0.8.0-alpha.0

Merge pull request #622 from 27149chen/update-featuregate-name

featureGate: IgnoreLeaseSync => IgnoreSyncLease

Clusterpedia v0.7.1

What's Changed

• [release/0.7] bump kwokctl by @wzshiming in #605

• [release/0.7] feat: added new parameters for parameterized query by @nekomeowww in #603
  Added three search queries - whereSQLStatement, whereSQLParam, and whereSQLJSONParams to make the internalstorage layer support parameterised sql. More detailed introduction to use

• [release/0.7] add featureGate IgnoreSyncLease to stop sync leases.coordination.k8s.io by @27149chen in #621 #623
  Added a feature gate - IgnoreSyncLease that does not synchronise the leases.coordination.k8s.io resource regardless of user configuration in PediaCluster

Full Changelog: v0.7.0...v0.7.1

Clusterpedia v0.7.1-rc.1

What's Changed

• [release/0.7] bump kwokctl by @wzshiming in #605
• [release/0.7] feat: added new parameters for parameterized query by @nekomeowww in #603
• [release/0.7] add featureGate IgnoreLeaseSync to stop sync leases.coordination.k8s.io by @27149chen in #621

Full Changelog: v0.7.0...v0.7.1-rc.1

Clusterpedia v0.7.1-rc.0

What's Changed

• [release/0.7] bump kwokctl by @wzshiming in #605
• [release/0.7] feat: added new parameters for parameterized query by @nekomeowww in #603

Full Changelog: v0.7.0...v0.7.1-rc.0