about CPU and memory usage of clickhouse-backup

[honobi]

1. Is there a way to limit the CPU and memory usage of clickhouse-backup?
2. Does the size of the data affect the CPU and memory usage of clickhouse-backup?
   I have currently set a CPU limit of 500m and a memory limit of 1G for the container where clickhouse-backup is running. I am concerned that high CPU or memory usage by clickhouse-backup may cause the pod to be terminated.

[Slach]

1. CPU/IO usage, it already have lowest nice and cpu nice priority, look https://github.com/Altinity/clickhouse-backup/?tab=readme-ov-file#explain-config-parameters cpu_nice_priority and io_nice_priority, memory usage could be control by upload_concurrency/download_concurrency settings in general config section and some concurrency settings specific for some remote storage types

2. yes, max size of data part is related look to SELECT max(bytes_on_disk)/4000 FROM system.parts WHERE database!='system'

if you have a lot of data you can use "small" container for backup, you will need to tune settings

[honobi]

1. What does max(bytes_on_disk)/4000 mean?

2. When the parameters upload_by_part and download_by_part are set to true, will clickhouse-backup load the entire part into memory? In other words, does my memory size need to be at least larger than the largest part?
   What should I do if a partition is so large that it cannot be loaded into memory?

3. When the upload_by_part and download_by_part parameters are set to false, how will the upload and download proceed?

[Slach]

What does max(bytes_on_disk)/4000 mean?

for s3 it means buffer size by default used for uploading/downloading minimum 2mb, max 5gb
read about data parts

https://clickhouse.com/docs/en/operations/system-tables/parts

When the parameters upload_by_part and download_by_part are set to true, will clickhouse-backup load the entire part into memory?

No, files inside part will read with streaming (some buffers for file IO linux page cache also should used) and compress (if compression_format is not none, better use tar, cause data in parts already compressed)
but will use different memory buffers with some size, depends on remote_store config parameter

different SDK use different buffers, look to https://github.com/Altinity/clickhouse-backup#explain-config-parameters, look concurrency for s3 or ftp

In other words, does my memory size need to be at least larger than the largest part?

this is not preciese
but for s3 is should not less
(upload_concurrency^2) * s3->concurency * max(bytes_on_disk) / s3->max_parts_count

but you can't predict bytes_on_disk

you run clickhouse-backup container in the same host with clickhouse-server
we prefer setup limits for whole kubernetes pod which contains clickhouse-backup+clickhouse-server container, instead of each container separately

there is backup memory usage graph for our typical 500Gb database last 7 days

500Gb database

upload_concurrency = 4
s3->concurrency = 17

formatReadableSize(max(bytes_on_disk/4000))─┐
24.72 MiB 

SELECT formatReadableSize(max(4*4*17*bytes_on_disk/4000)) FROM system.parts WHERE database!='system';

6.57 GiB                                                                                     │

this is maximum which never achieved