Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions tiproxy/tiproxy-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,39 @@ Configuration for SQL port.
+ Unit: second
+ When TiProxy shuts down, it closes connections when they have completed their current transactions (also known as draining clients) within `graceful-close-conn-timeout` seconds. After that, all the connections are closed at once. `graceful-close-conn-timeout` happens after `graceful-wait-before-shutdown`. It is recommended to set this timeout longer than the lifecycle of a transaction.

#### `fail-backend-list` <span class="version-mark">New in v1.3.3</span>

+ Default value: `[]`
+ Support hot-reload: yes
+ Specifies the list of backends to remove from routing. After confirming that a TiDB server has failed, you can add it to this list. TiProxy stops routing new connections to these backends and migrates existing connections away from them. Each item in the list can be in one of the following two forms:

- Backend Pod name, for example, `"db-tidb-0"`
- Backend address in the format of `<ip>:<port>`, for example, `"10.0.0.10:4000"`

+ If applying this list would leave no routable backends, TiProxy ignores this list to ensure that connection requests can still be routed.

#### `failover-timeout` <span class="version-mark">New in v1.3.3</span>

+ Default value: `60`
+ Hot-reload supported: Yes
+ Unit: second
+ Range: `>= 0`
+ When a backend appears in [`fail-backend-list`](#fail-backend-list-new-in-v133), TiProxy migrates existing connections away from that backend. If any connections remain on that backend after `failover-timeout` seconds, TiProxy forcibly closes these connections. `0` means that TiProxy forcibly closes any remaining connections immediately.

#### `max-connections`

+ Default value: `0`
+ Support hot-reload: yes
+ Each TiProxy instance can accept `max-connections` connections at most. `0` means no limitation.

#### `high-memory-usage-reject-threshold` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0.9`
+ Hot-reload supported: Yes
+ Range: `[0, 1]`
+ When the memory usage of TiProxy reaches or exceeds this threshold, TiProxy rejects new connections, and the status port returns an unhealthy status. Existing connections are not affected. If [`ha.virtual-ip`](#virtual-ip) is configured, the instance also releases the virtual IP. For example, `0.9` means that TiProxy starts rejecting new connections when the memory usage reaches 90%.
+ `0` means that TiProxy does not reject new connections based on memory usage. If the configured value is greater than `0` and less than `0.5`, TiProxy adjusts it to `0.5`.

#### `conn-buffer-size`

+ Default value: `32768`
Expand Down Expand Up @@ -128,6 +155,128 @@ Configurations for the load balancing policy of TiProxy.
+ Possible values: `resource`, `location`, `connection`
+ Specifies the load balancing policy. For the meaning of each possible value, see [TiProxy load balancing policies](/tiproxy/tiproxy-load-balance.md#configure-load-balancing-policies).

#### `routing-policy` <span class="version-mark">New in v1.3.3</span>

+ Default value: `prefer-idle`
+ Hot-reload supported: Yes
+ Possible values: `prefer-idle`, `random`, `idlest`
+ Specifies the routing policy for new connections:

- `prefer-idle`: Excludes backends that need connection migration, and then randomly selects from the remaining routable backends. Suitable for most scenarios.
- `random`: Randomly selects from routable backends, where the idlest backend has a slightly higher probability of being selected. Suitable for scenarios with a high rate of new connections.
- `idlest`: Always routes new connections to the idlest routable backend. Suitable for scenarios with long-lived connections and infrequent connection creation.

#### `status` <span class="version-mark">New in v1.3.3</span>

Status-based load balancing configuration.

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Range: `>= 0`
+ Specifies the number of connections migrated per second for status-based load balancing. `0` means that TiProxy automatically calculates the migration rate based on the current number of connections. When a TiDB server is shutting down, you can increase this value appropriately to speed up connection migration.

#### `health` <span class="version-mark">New in v1.3.3</span>

Health-based load balancing configuration. It takes effect only when [`policy`](#policy) is `resource` or `location`.

##### `enabled` <span class="version-mark">New in v1.3.3</span>

+ Default value: `true`
+ Hot-reload supported: Yes
+ Controls whether to enable [health-based load balancing](/tiproxy/tiproxy-load-balance.md#health-based-load-balancing).

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Range: `>= 0`
+ Specifies the number of connections migrated per second for health-based load balancing. `0` indicates that TiProxy automatically calculates the migration rate.

#### `memory` <span class="version-mark">New in v1.3.3</span>

Configuration for memory-based load balancing. This item takes effect only when [`policy`](#policy) is `resource` or `location`.

##### `enabled` <span class="version-mark">New in v1.3.3</span>

+ Default value: `true`
+ Hot-reload supported: Yes
+ Controls whether to enable [memory-based load balancing](/tiproxy/tiproxy-load-balance.md#memory-based-load-balancing).

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Range: `>= 0`
+ Specifies the number of connections migrated per second for memory-based load balancing. `0` indicates that TiProxy automatically calculates the migration rate.

#### `cpu` <span class="version-mark">New in v1.3.3</span>

Configuration for CPU-based load balancing. This item takes effect only when [`policy`](#policy) is `resource` or `location`.

##### `enabled` <span class="version-mark">New in v1.3.3</span>

+ Default value: `true`
+ Hot-reload supported: Yes
+ Controls whether to enable [CPU-based load balancing](/tiproxy/tiproxy-load-balance.md#cpu-based-load-balancing).

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Range: `>= 0`
+ Specifies the number of connections migrated per second for CPU-based load balancing. `0` indicates that TiProxy automatically calculates the migration rate. When CPU hotspots shift frequently, it is not recommended to set this value too high to avoid repeated connection migrations.

##### `min-balance-usage` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Range: `[0, 1]`
+ When the CPU usage of the source backend is lower than this threshold, CPU-based connection migration is not triggered. For example, `0.2` means no migration is performed when the CPU usage of the source backend is lower than 20%.

##### `max-usage-gap` <span class="version-mark">New in v1.3.3</span>

+ Default value: `1`
+ Hot-reload supported: Yes
+ Range: `0` or `[0.05, 1]`
+ Specifies the minimum CPU usage difference required to trigger CPU-based connection migration. Migration is triggered when the CPU usage difference between the source backend and the target backend reaches this threshold. For example, `0.1` means migration can be triggered when the difference reaches 10%. The default value `1` means whether to migrate depends only on adaptive rules. `0` means this parameter uses the default value. If you need more balanced CPU usage across backends, you can reduce this value appropriately.

#### `location` <span class="version-mark">New in v1.3.3</span>

Configuration for location-based load balancing. This item takes effect only when [`policy`](#policy) is `resource` or `location`.

##### `enabled` <span class="version-mark">New in v1.3.3</span>

+ Default value: `true`
+ Hot-reload supported: Yes
+ Controls whether to enable [location-based load balancing](/tiproxy/tiproxy-load-balance.md#location-based-load-balancing).

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Value range: `>= 0`
+ Specifies the number of connections migrated per second for location-based load balancing. `0` means the default migration rate is used.

#### `conn-count` <span class="version-mark">New in v1.3.3</span>

Configuration for connection-count-based load balancing.

##### `migrations-per-second` <span class="version-mark">New in v1.3.3</span>

+ Default value: `0`
+ Hot-reload supported: Yes
+ Value range: `>= 0`
+ Specifies the number of connections migrated per second for connection-count-based load balancing. `0` means TiProxy automatically calculates the migration rate. If you observe connections frequently migrating back and forth, you can reduce this value appropriately.

##### `count-ratio-threshold` <span class="version-mark">New in v1.3.3</span>

+ Default value: `1.2`
+ Hot-reload supported: Yes
+ Value range: `0` or `> 1`
+ Specifies the connection count ratio threshold for triggering connection-count-based migration. When the ratio of the backend with the most connections to the backend with the fewest connections exceeds this threshold, TiProxy starts migrating connections. Increasing this value can reduce migration frequency. `0` means this parameter uses the default value.

### ha

High availability configurations for TiProxy.
Expand Down Expand Up @@ -161,6 +310,20 @@ Starting from v1.3.1, TiProxy supports configuring multiple virtual IP addresses
+ Support hot-reload: no
+ Specifies the network interface to bind the virtual IP to, such as `"eth0"`. The virtual IP will be bound to a TiProxy instance only when both [`ha.virtual-ip`](#virtual-ip) and `ha.interface` are set.

#### `garp-burst-count` <span class="version-mark">New in v1.3.3</span>

+ Default value: `5`
+ Support hot-reload: no
+ Value range: `>= 0`
+ Specifies the number of GARP (Gratuitous ARP) packets sent immediately after a TiProxy instance takes over and binds the virtual IP. GARP is used to notify switches and hosts to update the MAC address corresponding to the virtual IP, so that client traffic can be switched to the TiProxy instance that has taken over the virtual IP as soon as possible. Sending multiple packets continuously can reduce the risk of switchover delay caused by the loss of the first GARP packet. `0` is automatically adjusted to `1`.

#### `garp-refresh-count` <span class="version-mark">New in v1.3.3</span>

+ Default value: `30`
+ Support hot-reload: no
+ Range: `>= 0`
+ Specifies the number of times to additionally send GARP bursts after taking over the virtual IP. The interval between two sends is 1 second, and [`garp-burst-count`](#garp-burst-count-new-in-v133) packets are sent each time. This is used to refresh the previous virtual IP-to-MAC address mapping in upstream devices for a period of time after failover, to avoid traffic still being forwarded to the old instance. `0` means no additional packets are sent after takeover.

### `labels`

+ Default value: `{}`
Expand Down
1 change: 1 addition & 0 deletions tiproxy/tiproxy-grafana.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ TiProxy has four panel groups. The metrics on these panels indicate the current
- proxy error: other TiProxy errors
- backend network break: fails to read from or write to the TiDB. This may be caused by a network problem or the TiDB server shutting down
- backend handshake fail: TiProxy fails to handshake with the TiDB server
- Connection Lifetime: the average and P99 connection lifetimes
- Goroutine Count: the number of Goroutines on each TiProxy instance
- Owner: the TiProxy instance that executes various tasks. For example, `10.24.31.1:3080 - vip` indicates that the TiProxy instance at `10.24.31.1:3080` is bound to a virtual IP. The tasks include the following:
- vip: binds a virtual IP
Expand Down
24 changes: 13 additions & 11 deletions tiproxy/tiproxy-load-balance.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,16 @@ TiProxy v1.0.0 only supports status-based and connection count-based load balanc

By default, TiProxy applies these policies with the following priorities:

1. Status-based load balancing: when a TiDB server is shutting down, TiProxy migrates connections from that TiDB server to an online TiDB server.
2. Label-based load balancing: TiProxy prioritizes routing requests to TiDB servers that share the same label as the TiProxy instance, enabling resource isolation at the computing layer.
1. Label-based load balancing: TiProxy prioritizes routing connection requests to TiDB servers that share the same label as the TiProxy instance, enabling resource isolation at the computing layer.
2. Status-based load balancing: when a TiDB server cannot provide service normally or is shutting down, TiProxy migrates connections from that TiDB server to an online TiDB server.
3. Health-based load balancing: when the health of a TiDB server is abnormal, TiProxy migrates connections from that TiDB server to a healthy TiDB server.
4. Memory-based load balancing: when a TiDB server is at risk of running out of memory (OOM), TiProxy migrates connections from that TiDB server to a TiDB server with lower memory usage.
5. CPU-based load balancing: when the CPU usage of a TiDB server is much higher than that of other TiDB servers, TiProxy migrates connections from that TiDB server to a TiDB server with lower CPU usage.
6. Location-based load balancing: TiProxy prioritizes routing requests to the TiDB server geographically closest to TiProxy.
6. Location-based load balancing: TiProxy prioritizes routing requests to TiDB servers that are geographically closer to TiProxy.
7. Connection count-based load balancing: when the connection count of a TiDB server is much higher than that of other TiDB servers, TiProxy migrates connections from that TiDB server to a TiDB server with fewer connections.

To adjust the priorities of load balancing policies, see [Configure load balancing policies](#configure-load-balancing-policies).

## Status-based load balancing

TiProxy periodically checks whether a TiDB server is offline or shutting down using the SQL port and status port.

## Label-based load balancing

Label-based load balancing prioritizes routing connections to TiDB servers that share the same label as TiProxy, enabling resource isolation at the computing layer. This policy is disabled by default and should only be enabled when your workload requires computing resource isolation.
Expand Down Expand Up @@ -100,6 +96,10 @@ pd_servers:
- host: pd-host-3
```

## Status-based load balancing

TiProxy periodically checks whether a TiDB server can provide services normally using the SQL port and status port, such as whether it is offline or shutting down.

## Health-based load balancing

TiProxy determines the health of a TiDB server by querying its error count. When the health of a TiDB server is abnormal while others are normal, TiProxy migrates connections from that server to a healthy TiDB server, achieving automatic failover.
Expand Down Expand Up @@ -189,7 +189,7 @@ In the preceding configuration, the TiProxy instance on `tiproxy-host-1` priorit

## Connection count-based load balancing

TiProxy migrates connections from a TiDB server with more connections to a server with fewer connections. This policy is not configurable and has the lowest priority.
TiProxy migrates connections from a TiDB server with more connections to a server with fewer connections. This policy has the lowest priority.

Typically, TiProxy identifies the load on TiDB servers based on CPU usage. This policy usually takes effect in the following scenarios:

Expand All @@ -200,9 +200,11 @@ Typically, TiProxy identifies the load on TiDB servers based on CPU usage. This

TiProxy lets you configure the combination and priority of load balancing policies through the [`policy`](/tiproxy/tiproxy-configuration.md#policy) configuration item.

- `resource`: the resource priority policy performs load balancing based on the following priority order: status, label, health, memory, CPU, location, and connection count.
- `location`: the location priority policy performs load balancing based on the following priority order: status, label, location, health, memory, CPU, and connection count.
- `connection`: the minimum connection count policy performs load balancing based on the following priority order: status, label, and connection count.
- `resource`: the resource priority policy performs load balancing based on the following priority order: label, status, health, memory, CPU, location, and connection count.
- `location`: the location priority policy performs load balancing based on the following priority order: label, status, location, health, memory, CPU, and connection count.
- `connection`: the minimum connection count policy performs load balancing based on the following priority order: label, status, and connection count.

For more configuration items related to load balancing, see [`balance`](/tiproxy/tiproxy-configuration.md#balance).

## More resources

Expand Down
4 changes: 2 additions & 2 deletions tiproxy/tiproxy-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ The following steps describe how to deploy TiProxy when creating a new cluster.

```yaml
component_versions:
tiproxy: "v1.3.2"
tiproxy: "v1.3.3"
server_configs:
tiproxy:
ha.virtual-ip: "10.0.1.10/24"
Expand Down Expand Up @@ -173,7 +173,7 @@ For clusters that do not have TiProxy deployed, you can enable TiProxy by scalin

```yaml
component_versions:
tiproxy: "v1.3.2"
tiproxy: "v1.3.3"
server_configs:
tiproxy:
ha.virtual-ip: "10.0.1.10/24"
Expand Down
Loading