Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch when all requests fail and make it clearer to the user why #213

Open
RyanMarten opened this issue Dec 5, 2024 · 0 comments
Open

Comments

@RyanMarten
Copy link
Contributor

          Ran into errors since my key didn't have gpt-4o access

We should catch that all of the requests failed and raise a relevant error instead of having the user exposed to the responses file

d with Exception API error: {'message': 'Project `proj_849B36OelpiRtrG7hEPPgVSD` does not have access to model `gpt-4o`', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}, attempts left 4
2024-12-04 18:44:30,733 - bespokelabs.curator.request_processor.base_online_request_processor - INFO - Adding request 0 to retry queue. Will retry in next available slot. Attempts remaining: 3
Processing OpenAIOnlineRequestProcessor requests:   0%|                                           | 0/10 [00:00<?, ?it/s]
2024-12-04 18:44:30,736 - bespokelabs.curator.request_processor.base_online_request_processor - INFO - Processing complete. Results saved to /Users/ryan/.cache/curator/f467067fea2470e9/responses_0.jsonl
2024-12-04 18:44:30,737 - bespokelabs.curator.request_processor.base_online_request_processor - INFO - Status tracker: Tasks - Started: 10, In Progress: 10, Succeeded: 0, Failed: 0, Already Completed: 0
Errors - API: 20, Rate Limit: 0, Other: 20, Total: 40
Traceback (most recent call last):
  File "/Users/ryan/curator/examples/persona-hub/synthesize.py", line 75, in <module>
    main(args)
  File "/Users/ryan/curator/examples/persona-hub/synthesize.py", line 46, in main
    output = generator(persona_dataset)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ryan/curator/src/bespokelabs/curator/prompter/prompter.py", line 191, in __call__
    return self._completions(self._request_processor, dataset, working_dir)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ryan/curator/src/bespokelabs/curator/prompter/prompter.py", line 279, in _completions
    dataset = request_processor.run(
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ryan/curator/src/bespokelabs/curator/request_processor/base_online_request_processor.py", line 177, in run
    return self.create_dataset_files(working_dir, parse_func_hash, prompt_formatter)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ryan/curator/src/bespokelabs/curator/request_processor/base_request_processor.py", line 200, in create_dataset_files
    raise ValueError(f"No responses files found in {working_dir}")
ValueError: No responses files found in /Users/ryan/.cache/curator/f467067fea2470e9

Originally posted by @RyanMarten in #211 (review)

devin-ai-integration bot added a commit that referenced this issue Dec 5, 2024
- Add specific handling for model access errors in OpenAIOnlineRequestProcessor
- Provide clearer error messages with actionable information
- Enhance 'all requests failed' error message with better context
- Fixes #213
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant