feat(cmd): Utility to import / export chain data

[renaynay]

Description

A utility to import/export chain data is very necessary for testing + potentially useful for ops (see https://github.com/celestiaorg/devops/issues/35).

Ideally it could look something like:

celestia bridge import-chain <path to blocks directory>

celestia bridge export-chain

This would likely be a larger undertaking as we have a separation between headers + their corresponding block data. I have not had time yet to scope out this issue, but just dumping my thoughts here as to not forget.

[smuu]

It would be nice for automation not to need to run commands. For example, for celestia-app, it seems it is possible that you can simply copy the data folder to a new instance and solve quicksync this way.

[renaynay]

I would prefer that this action is explicit via commands.

[musalbas]

What would the command do, would it need to re-import everything into badger or copy and paste the data again? That would defeat the idea of it being a fastsync

[Wondertan]

Copy pasting is enought

[musalbas]

Copy pasting terabytes of data on disk will be slow and there might not be enough disk space for it as people would need 2x the disk space, compared to just copying the data folder directly

[Wondertan]

Ok, I misunderstood your question then. I am not sure there is something else we can do in import/export cmds besides copying.

[musalbas]

My comment is in a response to:

I would prefer that this action is explicit via commands.

I do not think it makes sense as a command, we should just support it as copying data folder:

It would be nice for automation not to need to run commands. For example, for celestia-app, it seems it is possible that you can simply copy the data folder to a new instance and solve quicksync this way.

[Wondertan]

we should just support it as copying data folder

Then I understand your question right. It's currently supported. You can copy folders without calling any special commands to achieve this. This issue is more about eX around it

Discussions around optimizing copying should be a separate issue, likely in devops repo. We may have only a single copy of data over high-bandwidth storage system, which multiple nodes mount to. Either through dagstore mounts or using NFSes or similar

[renaynay]

Closing as non-node-related.

[musalbas]

Why non-node related?