Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement logits_filter_callback #193

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

githubnemo
Copy link

This is exposed via on_new_logits callback in the Python API to be consistent with the on_new_... scheme.

Note that currently only the number of tokens so far and the raw logits are exposed. So to be truly compatible with whisper.cpp the other arguments would need to be exposed to Python as well but it is better than nothing.

The logits are mutable so that the callback can indeed influence the decoding process. By using the array_t type this also safes us a copy.

I've added a test but I was not able to run the tests locally:

% ./tools/bazel test tests/...                                                                                                              (git)-[feature/logits-callback] 
INFO: Running bazel wrapper (see //tools/bazel for details), bazel version 6.0.0 will be used instead of system-wide bazel installation.
ERROR: .../pywhispercpp/tests/BUILD:26:8: no such package '@pypi_pytest_io//': The repository '@pypi_pytest_asyncio' could not be resolved: Repository '@pypi_pytest_asyncio' is not defined and referenced by '//tests:utils'
ERROR: Analysis of target '//tests:utils' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.107s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
ERROR: Couldn't start the build. Unable to run tests

This is exposed via `on_new_logits` callback in the Python API
to be consistent with the `on_new_...` scheme.

Note that currently only the number of tokens so far and the
raw logits are exposed. So to be truly compatible with whisper.cpp
the other arguments would need to be exposed to Python as well
but it is better than nothing.

The logits are mutable so that the callback can indeed influence the
decoding process. By using the `array_t` type this also safes us a copy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant