Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add temperature and top-p #77

Merged
merged 3 commits into from
Nov 13, 2024
Merged

Add temperature and top-p #77

merged 3 commits into from
Nov 13, 2024

Conversation

RyanMarten
Copy link
Contributor

Doing this since Negin requires it for UnnaturalInstructions.

Also to get an idea of the best way to do #62 and #74.

This is just a stop gap before I get to those two issues.

@RyanMarten RyanMarten changed the base branch from main to dev November 13, 2024 00:26
Comment on lines 53 to 54
temperature: float = 1.0,
top_p: float = 1.0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the right defaults to use?
Should it not be None?
https://docs.litellm.ai/docs/completion/input:

    temperature: Optional[float] = None,

If it is None, I assume litellm doesn't pass the temperature and the provider will the default that they have configured.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing this!

Copy link
Contributor

@CharlieJCJ CharlieJCJ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@RyanMarten
Copy link
Contributor Author

Test with

from bespokelabs.curator import Prompter
from datasets import Dataset

dataset = Dataset.from_dict({"prompt": ["write me a poem"] * 10})

prompter = Prompter(
    prompt_func=lambda row: row["prompt"],
    model_name="gpt-4o-mini",
    response_format=None,
    temperature=TEMP,
)

dataset = prompter(dataset)
print(dataset.to_pandas())

Temperature 0.0

0  In the hush of dawn's embrace,  \nWhere whispe...
1  In the hush of dawn's embrace,  \nWhere whispe...
2  In the hush of dawn's embrace,  \nWhere whispe...
3  In the hush of dawn's embrace,  \nWhere whispe...
4  In the hush of dawn's embrace,  \nWhere whispe...
5  In the hush of dawn's embrace,  \nWhere whispe...
6  In the hush of dawn's embrace,  \nWhere whispe...
7  In the hush of dawn's embrace,  \nWhere whispe...
8  In the hush of dawn's embrace,  \nWhere whispe...
9  In the hush of dawn's embrace,  \nWhere whispe...

Temperature 1.5

0  In the hush of morning's light,  \nWhere whisp...
1  In the quiet hush of dawning light,  \nThe wor...
2  In a quiet grove where violets dance,  \nBenea...
3  In twilight’s hush, when shadows play,  \nThe ...
4  In a garden where the blossoms sing,  \nBeneat...
5  Beneath the sky, in twilight hues,  \nA whispe...
6  In the hush of dawn's embrace,  \nA gentle whi...
7  In the garden where the wildflowers sway,  \nU...
8  In a quiet glade where shadows dwell,  \nWhisp...
9  In the quiet of the dawn, when shadows blend w...

@RyanMarten RyanMarten merged commit 3b32c1f into dev Nov 13, 2024
@RyanMarten RyanMarten deleted the ryanm/temperature-and-top-p branch November 13, 2024 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants