Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compression algorithm should be none or lz4_block #1644

Open
wswsmao opened this issue Nov 4, 2024 · 3 comments
Open

compression algorithm should be none or lz4_block #1644

wswsmao opened this issue Nov 4, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@wswsmao
Copy link
Contributor

wswsmao commented Nov 4, 2024

I get error like this.
according to
https://github.com/dragonflyoss/nydus/blob/master/docs/chunk-deduplication.md

$ RUST_BACKTRACE=1 nydusify chunkdict generate --sources  abushwang/redis:7.2.6-nydusify,abushwang/redis:7.4-nydusify,abushwang/redis:7.4.1-nydusify --target abushwang/redis:nydus_chunkdict --source-insecure --target-insecure
INFO[2024-11-04T02:51:53Z] Parsing image abushwang/redis:7.2.6-nydusify 
INFO[2024-11-04T02:51:59Z] Pulling Nydus bootstrap to output/abushwang:redis:7.2.6-nydusify/nydus_bootstrap 
INFO[2024-11-04T02:52:01Z] Parsing image abushwang/redis:7.4-nydusify   
INFO[2024-11-04T02:52:07Z] Pulling Nydus bootstrap to output/abushwang:redis:7.4-nydusify/nydus_bootstrap 
INFO[2024-11-04T02:52:10Z] Parsing image abushwang/redis:7.4.1-nydusify 
INFO[2024-11-04T02:52:16Z] Pulling Nydus bootstrap to output/abushwang:redis:7.4.1-nydusify/nydus_bootstrap 
INFO[2024-11-04T02:52:18Z] Invoking 'nydus-image chunkdict generate' command 
Warning: Blob with id '33b39229606897a7712995a726e992d8d96ce864a5c2c8e6a3b19e79b7bee53e' is smaller than 4096 bytes.
[2024-11-04 02:53:37.862528 +00:00] ERROR [/src/error.rs:18] Stack:
   0: nydus_api::error::make_error

[2024-11-04 02:53:37.862586 +00:00] ERROR [/src/error.rs:19] Error:
        "compression algorithm should be none or lz4_block"
        at utils/src/compress/mod.rs:45
Error: Invalid argument (os error 22)
ERRO[2024-11-04T02:53:37Z] fail to run nydus-image [chunkdict generate --log-level warn --bootstrap output/chunkdict_bootstrap --database sqlite:///home/abushwang/nydus-static-2.3.0/output/database.db --output-json output/nydus_bootstrap_output.json output/abushwang:redis:7.2.6-nydusify/nydus_bootstrap output/abushwang:redis:7.4-nydusify/nydus_bootstrap output/abushwang:redis:7.4.1-nydusify/nydus_bootstrap]  error="exit status 1"
FATA[2024-11-04T02:53:37Z] invalid nydus bootstrap format: exit status 1 

I get nydusify images by this command:

$ nydusify convert --source abushwang/redis:$source_tag --target abushwang/redis:$source_tag-nydusify

nydusify comes from this:

$ wget https://github.com/dragonflyoss/nydus/releases/download/v2.3.0/nydus-static-v2.3.0-linux-amd64.tgz
@wswsmao
Copy link
Contributor Author

wswsmao commented Nov 4, 2024

I have add --compressor, but not work

$ nydusify convert --compressor none --source abushwang/redis:$source_tag --target abushwang/redis:$source_tag-nfy_lz

...

$ nydusify convert --compressor lz4_block --source abushwang/redis:$source_tag --target abushwang/redis:$source_tag-nfy_lz

@imeoer
Copy link
Collaborator

imeoer commented Nov 4, 2024

@cslinwang PTAL.

@imeoer imeoer added the bug Something isn't working label Nov 4, 2024
@Jaycean
Copy link

Jaycean commented Dec 3, 2024

I also have the same problem #1651

[root@master ~]# nydusify chunkdict generate --sources 192.168.237.131:80/library/ubuntu:20.04-nydus --sources 192.168.237.131:80/library/ubuntu:22.04-nydus --target 192.168.237.131:80/library/ubuntu:dict3 --source-insecure --target-insecure
INFO[2024-12-03T00:52:11-08:00] Parsing image 192.168.237.131:80/library/ubuntu:20.04-nydus 
INFO[2024-12-03T00:52:11-08:00] trying next host                              error="failed to do request: Head \"https://192.168.237.131:80/v2/library/ubuntu/manifests/20.04-nydus\": http: server gave HTTP response to HTTPS client" host="192.168.237.131:80"
INFO[2024-12-03T00:52:11-08:00] Parsing image 192.168.237.131:80/library/ubuntu:20.04-nydus 
INFO[2024-12-03T00:52:11-08:00] Pulling Nydus bootstrap to output/192.168.237.131:80:library:ubuntu:20.04-nydus/nydus_bootstrap 
INFO[2024-12-03T00:52:11-08:00] Parsing image 192.168.237.131:80/library/ubuntu:22.04-nydus 
INFO[2024-12-03T00:52:12-08:00] Pulling Nydus bootstrap to output/192.168.237.131:80:library:ubuntu:22.04-nydus/nydus_bootstrap 
INFO[2024-12-03T00:52:12-08:00] Invoking 'nydus-image chunkdict generate' command 
[2024-12-03 00:52:17.997449 -08:00] ERROR [/src/error.rs:18] Stack:
   0: nydus_api::error::make_error

[2024-12-03 00:52:17.997559 -08:00] ERROR [/src/error.rs:19] Error:
        "compression algorithm should be none or lz4_block"
        at utils/src/compress/mod.rs:45
Error: Invalid argument (os error 22)
ERRO[2024-12-03T00:52:17-08:00] fail to run nydus-image [chunkdict generate --log-level warn --bootstrap output/chunkdict_bootstrap --database sqlite:///root/output/database.db --output-json output/nydus_bootstrap_output.json output/192.168.237.131:80:library:ubuntu:20.04-nydus/nydus_bootstrap output/192.168.237.131:80:library:ubuntu:22.04-nydus/nydus_bootstrap]  error="exit status 1"
FATA[2024-12-03T00:52:17-08:00] invalid nydus bootstrap format: exit status 1 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants