Add rate limiter for OpenAI API call

[Savaw]

Added a rate_limit function that can be used with OpenAI API calls to prevent "Rate Limit Reached" error.

[keerthanpg]

What is the default rate limit of open ai ? Wouldn't change make the app slower?

[Savaw]

The default rate limit for free users is 20 rpm, as mentioned on the website. However, the error I faced yesterday said I have exceeded the limit of 60 rpm. (I was trying a paper with more than 60 rows in its dataframe.) It can make the app slower, but it is better than not working. The default min_interval_seconds can also be changed to change the speed according to rate limit.

Anyway, another option that I saw on the above link today and might be better would be using a retry mechanism, so the code performs normally at the beginning and slows down only if faced with an error:

from tenacity import retry, stop_after_attempt, wait_random_exponential
 
@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
def get_embedding_with_backoff(*args, **kwargs):
    return get_embedding(*args, **kwargs)