Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Python interface #817

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

add Python interface #817

wants to merge 5 commits into from

Conversation

Picus303
Copy link
Contributor

This PR allows to use Fish Speech as an importable Python library for easy integration in code.
It adds the sub-directory lib in the main directory and mainly relies on the dedicated Pipeline class.

Here is an example of use:

# Optionnal: remove prints in terminal
import os
from loguru import logger

logger.remove()
os.environ["TQDM_DISABLE"] = "1"

# Prepare the models
from fish_speech.lib import Pipeline

model = Pipeline(
    llama_path = "models/fish-speech-1_5",
    vqgan_path = "models/vqgan-1_5.pth",
)

# Create a reference audio
ref = model.make_reference("ref.wav", "reference text")

# Generate audio (no streaming)
output = model.generate("text to generate.", ref)

# Generate audio (streaming)
import numpy as np

generator = model.generate("text to generate.", ref, streaming=True)

parts = []
for part in generator:
    parts.append(part)
    print(part.shape)

output = np.concatenate(parts, axis=0)

# Save the output to a file
sample_rate = model.sample_rate

import soundfile as sf
sf.write("output.wav", output, sample_rate)

This PR is not complete yet as it's missing:
1 - Documentation. Question: Where do you want to put it?
2 - The code still depends on .project-root to manage paths, making it impossible to install in non-editable mode, forcing the user to keep the source code in a separate folder. I'd be glad if you have suggestions for this part.

@Picus303 Picus303 marked this pull request as draft January 10, 2025 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant