Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: fix chunk map compatibility #1417

Merged
merged 1 commit into from
Sep 5, 2023

Conversation

imeoer
Copy link
Collaborator

@imeoer imeoer commented Sep 4, 2023

The blob cache file of nydusd v2.2 and <=v2.1 are in different
formats, which are not compatible. Should use different chunk map
files for them, in order to upgrade or downgrade smoothly.

For the nydusd <=v2.1, the files in blob cache directory:

$blob_id
$blob_id.chunk_map

For the nydusd =v2.2, the files in blob cache directory:

$blob_id.blob.data
$blob_id.chunk_map

NOTE: nydusd (v2.2) maybe use the chunk map file of nydusd(<=v2.1),
it will cause the corrupted blob cache data to be read.

For the nydusd of current patch, the files in blob cache directory:

$blob_id.blob.data
$blob_id.blob.data.chunk_map

NOTE: this will discard the old blob cache data and chunk map files.

Types of changes

What types of changes does your PullRequest introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@imeoer imeoer requested a review from a team as a code owner September 4, 2023 10:26
@imeoer imeoer requested review from luodw, gaius-qi and changweige and removed request for a team September 4, 2023 10:26
The blob cache file of nydusd v2.2 and <=v2.1 are in different
formats, which are not compatible. Should use different chunk map
files for them, in order to upgrade or downgrade smoothly.

For the nydusd <=v2.1, the files in blob cache directory:

```
$blob_id
$blob_id.chunk_map
```

For the nydusd =v2.2, the files in blob cache directory:

```
$blob_id.blob.data
$blob_id.chunk_map
```

NOTE: nydusd (v2.2) maybe use the chunk map file of nydusd(<=v2.1),
it will cause the corrupted blob cache data to be read.

For the nydusd of current patch, the files in blob cache directory:

```
$blob_id.blob.data
$blob_id.blob.data.chunk_map
```

NOTE: this will discard the old blob cache data and chunk map files.

Signed-off-by: Yan Song <[email protected]>
@imeoer imeoer force-pushed the fix-chunkmap-compatibility branch from baeea7f to 5e506f4 Compare September 4, 2023 10:28
@imeoer imeoer changed the title fix: compatible to old chunk map for nydusd storage: fix chunk map compatibility Sep 4, 2023
@codecov
Copy link

codecov bot commented Sep 4, 2023

Codecov Report

Merging #1417 (5e506f4) into master (631db29) will decrease coverage by 0.02%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1417      +/-   ##
==========================================
- Coverage   46.45%   46.44%   -0.02%     
==========================================
  Files         123      123              
  Lines       38649    38643       -6     
  Branches    38649    38643       -6     
==========================================
- Hits        17956    17948       -8     
+ Misses      19724    19721       -3     
- Partials      969      974       +5     
Files Changed Coverage Δ
storage/src/cache/fscache/mod.rs 0.00% <0.00%> (ø)
storage/src/cache/state/indexed_chunk_map.rs 83.33% <ø> (+2.03%) ⬆️
utils/src/metrics.rs 33.25% <ø> (-0.67%) ⬇️
storage/src/cache/filecache/mod.rs 66.54% <75.00%> (ø)

... and 2 files with indirect coverage changes

@imeoer
Copy link
Collaborator Author

imeoer commented Sep 5, 2023

The compatibility issue was found by @ccx1024cc and @power-more, they provided a repair idea.

@jiangliu jiangliu merged commit 1d93f12 into dragonflyoss:master Sep 5, 2023
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants