Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: write to rocksdb by our Batch interface (hincrby hmset hsetnx) #350

Merged

Conversation

longfar-ncy
Copy link
Collaborator

@longfar-ncy longfar-ncy commented Jun 12, 2024

issure: #280

Summary by CodeRabbit

  • 测试
    • 添加了一系列一致性测试,用于验证Redis命令(如HSetnxHMSetHIncrbyHIncrbyfloatSAddSRem)的操作一致性和正确性。
  • 重构
    • 改进了Redis类中的批处理操作,提升了写操作的抽象层次,可能提高了性能和可靠性。

Copy link

coderabbitai bot commented Jun 12, 2024

Walkthrough

这些变更主要涉及将 Redis 命令的操作从直接使用 rocksdb::WriteBatch 转换为使用封装的 Batch 接口。具体包括修改 HIncrbyHIncrbyfloatHMSetHSetnx 四个命令,以及为这些命令添加相应的一致性测试,以确保在批量操作时保持数据的一致性和正确性。

Changes

文件 修改摘要
src/storage/src/redis_hashes.cc rocksdb::WriteBatch 替换为 Batch::CreateBatch 并修改写入逻辑以使用新接口。
tests/consistency_test.go 增加了 HSetnxHMSetHIncrbyHIncrbyfloat 的一致性测试;修改了 SAdd & SRem 的测试以包括一致性检查。

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant RedisServer
    participant Batch

    Client->>RedisServer: 请求 HIncrby/HIncrbyfloat/HMSet/HSetnx
    RedisServer->>Batch: 使用 Batch::CreateBatch 创建批处理
    Batch-->>RedisServer: 返回 Batch 对象
    RedisServer->>Batch: 执行 put 和 delete 操作
    RedisServer->>Batch: 调用 Batch 的 Commit 方法
    Batch-->>RedisServer: 返回 Commit 结果
    RedisServer-->>Client: 发送操作结果
Loading

Possibly related issues

Poem

在代码的海洋中寻梦,
批处理接口现身跃鱼龙。
增加测试守护一致性,
Redis 栈中万变皆从容。
🐇🌟批代码,迎新风。


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the ✏️ Feature New feature or request label Jun 12, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between eec3030 and fb47b78.

Files selected for processing (2)
  • src/storage/src/redis_hashes.cc (17 hunks)
  • tests/consistency_test.go (1 hunks)
Additional comments not posted (8)
tests/consistency_test.go (4)

156-175: 测试用例“HSetnx 一致性测试”实现了对 HSetNX 命令的一致性检查。该测试首先设置一个值,然后尝试设置相同字段的新值,并验证后续尝试不会覆盖现有值。读取检查确保值在多次读取中保持一致。此测试用例正确地实现了预期的检查。


177-195: 测试用例“HMSet 一致性测试”实现了对 HMSet 命令的一致性检查。该测试设置多个字段-值对,并在多次读取中检查它们的一致性。此测试用例正确地实现了预期的检查。


197-237: 测试用例“HIncrby 一致性测试”实现了对 HIncrby 命令的一致性检查。该测试正确地增加了字段的值,并在多次读取中检查增加后的值的一致性。此测试用例正确地实现了预期的检查。


239-282: 测试用例“HIncrbyfloat 一致性测试”实现了对 HIncrbyfloat 命令的一致性检查。该测试正确地以浮点数增加了字段的值,并在多次读取中检查增加后的值的一致性。此测试用例正确地实现了预期的检查。

src/storage/src/redis_hashes.cc (4)

Line range hint 287-357: 优化了批处理操作的实现方式,使用 Batch::CreateBatch(this) 替代了直接使用 rocksdb::WriteBatch,这有助于提高代码的可维护性和可扩展性。


Line range hint 685-734: HSetnx 方法的修改也是为了使用新的批处理接口。这是一个重要的改进,因为它关系到数据的一致性和原子性。建议进行彻底的测试,确保在并发环境下的行为正确无误。

Verification successful

HSetnx 方法在 tests/consistency_test.go 文件中确实有测试覆盖。测试内容包括在领导节点上执行 HSetNX 操作,并检查其结果是否符合预期。此外,还包括对从节点的读取检查,以确保数据一致性。

  • tests/consistency_test.go 文件中有一个名为 "HSetnx Consistency Test" 的测试用例,验证了 HSetNX 方法的基本功能和一致性。

以下是需要注意的代码位置:

  • tests/consistency_test.go 文件中的 "HSetnx Consistency Test" 测试用例。
Scripts executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify thorough testing of `HSetnx` to ensure correct behavior under concurrent conditions.
# Test: Search for tests involving `HSetnx`.
rg --type go $'func Test.*HSetnx.*'

Length of output: 36



Script:

#!/bin/bash
# Description: Verify thorough testing of `HSetnx` to ensure correct behavior under concurrent conditions.
# Test: Search for tests involving `HSetnx` in all possible test files and formats.

# Search in Go test files
rg --type go 'HSetnx'

# Search in C++ test files
rg --type cpp 'HSetnx'

# Search in Python test files
rg --type py 'HSetnx'

# Search in any other test files
rg --type test 'HSetnx'

Length of output: 972



Script:

#!/bin/bash
# Description: Display the content of the Go test file that references `HSetnx` to verify thorough testing.
cat tests/consistency_test.go

Length of output: 28642


Line range hint 556-614: HMSet 方法的改动也是将 rocksdb::WriteBatch 替换为 Batch::CreateBatch(this)。这种改动有助于统一批处理操作的实现,但需要注意是否所有相关的调用都已经适配这种新的批处理方式。


Line range hint 364-438: 同样,HIncrbyfloat 方法也采用了新的批处理接口。请确保所有相关的单元测试都已更新,以覆盖这些更改。

@AlexStocks
Copy link
Contributor

CI 失败 fix 下

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


CI failed fix next

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between fb47b78 and 74e1d56.

Files selected for processing (1)
  • tests/consistency_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • tests/consistency_test.go

@Tangruilin Tangruilin marked this pull request as draft July 13, 2024 11:43
@longfar-ncy longfar-ncy changed the title feat: write to rocksdb by our Batch interface (hincrby hincrbyfloat hmset hsetnx) feat: write to rocksdb by our Batch interface (hincrby hmset hsetnx) Jul 20, 2024
@longfar-ncy longfar-ncy marked this pull request as ready for review July 27, 2024 04:30
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 74e1d56 and 3bed003.

Files selected for processing (2)
  • src/storage/src/redis_hashes.cc (13 hunks)
  • tests/consistency_test.go (2 hunks)
Additional comments not posted (7)
tests/consistency_test.go (4)

156-175: 测试用例通过: HSetnx 一致性测试

该测试用例结构良好,覆盖了 HSetNX 的预期行为。


177-195: 测试用例通过: HMSet 一致性测试

该测试用例结构良好,覆盖了 HMSet 的预期行为。


197-237: 测试用例通过: HIncrby 一致性测试

该测试用例结构良好,覆盖了 HIncrBy 的预期行为。


907-907: 优化: 减少 readChecker 函数中的休眠时间

将休眠时间从 10 秒减少到 1 秒,可能旨在优化测试执行时间,同时不影响读取检查的完整性。

src/storage/src/redis_hashes.cc (3)

Line range hint 287-357: 重构通过: HIncrby 方法

该方法已重构为使用 Batch 类,增强了批处理操作的抽象。请确保批处理操作的正确性。


Line range hint 556-614: 重构通过: HMSet 方法

该方法已重构为使用 Batch 类,增强了批处理操作的抽象。请确保批处理操作的正确性。


Line range hint 685-734: 重构通过: HSetnx 方法

该方法已重构为使用 Batch 类,增强了批处理操作的抽象。请确保批处理操作的正确性。

@AlexStocks AlexStocks merged commit c8e0cde into OpenAtomFoundation:unstable Jul 27, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✏️ Feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants