Skip to content

Commit

Permalink
Fix typo in readme and remove the ToArgilla step (#548)
Browse files Browse the repository at this point in the history
* Fix typo in readme and remove the ToArgilla step

* Update README.md

* Update index.md

* Update README.md
  • Loading branch information
dvsrepo committed Apr 17, 2024
1 parent 7bbc8ee commit ffc0f4f
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 32 deletions.
24 changes: 5 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,18 +91,18 @@ In addition, the following extras are available:

### Example

To run the following example you must install `distilabel` with both `openai` and `argilla` extras:
To run the following example you must install `distilabel` with both `openai` extra:

```sh
pip install "distilabel[openai,argilla]" --upgrade
pip install "distilabel[openai]" --upgrade
```

Then run the following example:
Then run:

```python
from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps import LoadHubDataset, TextGenerationToArgilla
from distilabel.steps import LoadHubDataset
from distilabel.steps.tasks import TextGeneration

with Pipeline(
Expand All @@ -115,14 +115,10 @@ with Pipeline(
)

generate_with_openai = TextGeneration(
name="generate_with_gpt3.5", llm=OpenAILLM(model="gpt-3.5-turbo")
name="generate_with_gpt35", llm=OpenAILLM(model="gpt-3.5-turbo")
)

to_argilla = TextGenerationToArgilla(name="to_argilla")

load_dataset.connect(generate_with_openai)
generate_with_openai.connect(to_argilla)


if __name__ == "__main__":
distiset = pipeline.run(
Expand All @@ -139,20 +135,10 @@ if __name__ == "__main__":
}
}
},
"to_argilla": {
"dataset_name": "text-generations-with-gpt35",
"dataset_workspace": "admin",
},
},
)
distiset.push_to_hub(
"distilabel-internal-testing/instruction-dataset-mini-with-generations"
)
```

Distilabel integrates smoothly with Argilla and provides all the necessary configurations to make giving a final human touch as easy as possible.


## Badges

If you build something cool with `distilabel` consider adding one of these badges to your dataset or model card.
Expand Down
23 changes: 10 additions & 13 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,18 @@ In addition, the following extras are available:

## Quick example

To run the following example you must install `distilabel` with both `openai` extra:

```sh
pip install "distilabel[openai]" --upgrade
```

Then run:

```python
from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps import LoadHubDataset, TextGenerationToArgilla
from distilabel.steps import LoadHubDataset
from distilabel.steps.tasks import TextGeneration

with Pipeline(
Expand All @@ -50,14 +58,10 @@ with Pipeline(
)

generate_with_openai = TextGeneration(
name="generate_with_gpt3.5", llm=OpenAILLM(model="gpt-3.5-turbo")
name="generate_with_gpt35", llm=OpenAILLM(model="gpt-3.5-turbo")
)

to_argilla = TextGenerationToArgilla(name="to_argilla")

load_dataset.connect(generate_with_openai)
generate_with_openai.connect(to_argilla)


if __name__ == "__main__":
distiset = pipeline.run(
Expand All @@ -74,13 +78,6 @@ if __name__ == "__main__":
}
}
},
"to_argilla": {
"dataset_name": "text-generations-with-gpt35",
"dataset_workspace": "admin",
},
},
)
distiset.push_to_hub(
"distilabel-internal-testing/instruction-dataset-mini-with-generations"
)
```

0 comments on commit ffc0f4f

Please sign in to comment.