Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Allow setting encode kwargs in SentenceBert embedder #80

Merged
merged 3 commits into from
Nov 27, 2024

Conversation

lsz05
Copy link
Collaborator

@lsz05 lsz05 commented Sep 27, 2024

関連する Issue / PR

#77

PR をマージした後の挙動の変化

encode時に独自な引数が必要な場合を対応

挙動の変更を達成するために行ったこと

  • embedderのencode関数にencode_kwargsを追加
  • 各タスクのevaluatorもencode_kwargs引数対応

動作確認

  • テストが通ることを確認した
  • マージ先がdevブランチであることを確認した

@lsz05 lsz05 marked this pull request as draft September 27, 2024 08:19
@lsz05 lsz05 changed the base branch from main to dev September 27, 2024 08:19
@lsz05 lsz05 mentioned this pull request Sep 27, 2024
17 tasks
@lsz05 lsz05 marked this pull request as ready for review November 27, 2024 04:41
@lsz05 lsz05 requested a review from akiFQC November 27, 2024 04:41
@akiFQC
Copy link
Collaborator

akiFQC commented Nov 27, 2024

全体的にLGTMですが、kwarfgsに検索するdoc stringを足していただけると良いかと思いました

@@ -43,6 +43,7 @@ def _batch_encode_and_save_on_disk(
prefix: str | None = None,
batch_size: int = 262144,
dtype: str = "float32",
**kwargs,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kwargについて、簡単な説明を加えてください。

@lsz05 lsz05 merged commit 0ded6ad into dev Nov 27, 2024
3 checks passed
@lsz05 lsz05 mentioned this pull request Dec 11, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants