Create remote backup via scp

[gavishdewangan]

Is there a provision of creating remote backup via scp, I could find only sftp as option, is there any workaround for scp?

[Slach]

Why sftp is not allowed for you? Did you try it?
Usually this is should be the same with scp

if not
you can create
list.sh
delete.sh
upload.sh
download.sh

and use scp inside

maybe rsync will more usefull for you
look examples in
https://github.com/Altinity/clickhouse-backup/tree/master/test/integration/rsync
and
https://github.com/Altinity/clickhouse-backup/blob/master/test/integration/config-custom-rsync.yml

[gavishdewangan]

A follow up question regarding this, in the above example it is copying existing backup to remote server, for that the backup needs to exist first, is there a way to create and store backup on the fly

[Slach]

this is different question
we decided split backup to main 4 isolated commands

• create
• upload
• download
• restore

why do you need "on the fly" approach? which goal you try to achieve?

[gavishdewangan]

I was thinking of reducing number operations. In the rsync example we have upload and download, for create and restore how are we doing that in this particular scenario?

[Slach]

could you read and figure out links shared above?

• create execute ALTER TABLE ... FREEZE ... and use file system hard links and don't allocates disk space
  this is universal operation for all storage types
• upload.sh execute rsync to upload from /var/lib/clickhouse/backup/backup_name to your remote storage
• download.sh execute rsync to download from your remote storage to /var/lib/clickhouse/backup/backup_name, and this operation allocates disk space on destination server
• restore execute ALTER TABLE ... ATTACH PART, and it also use hardlinks and not allocate additional space after download