Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto polygon has paid #476

Merged
merged 4 commits into from
Jun 30, 2024
Merged

Auto polygon has paid #476

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
59451fe
made polygon_has_paid_subscription automatic and added polygon promot…
grzesir Jun 29, 2024
dc15e18
colored the warning
grzesir Jun 29, 2024
34090c0
version bump
grzesir Jun 29, 2024
1e9b851
fixed test
grzesir Jun 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Lumibot - Algorithmic Trading Library
# Lumibot - A Backtesting and Trading Library for Stocks, Options, Crypto, Futures and More!

Backtesting and trading for stocks, options, crypto, futures and more!
Lumibot is a backtesting and trading library for stocks, options, crypto, futures and more. It is made so that the same code you use for backtesting can be used for live trading, making it easy to transition from backtesting to live trading. Lumibot is a highly flexible library that allows you to create your own strategies and indicators, and backtest them on historical data. It is also highly optimized for speed, so you can backtest your strategies quickly and efficiently.

**IMPORTANT: This library requires data for backtesting. The recommended data source is Polygon, and you can get an API key at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 (a free tier is available too) Please use the full link to give us credit for the sale, it helps support this project. You can use the coupon code 'LUMI10' for 10% off.**

## Documentation - 👇 Start Here 👇

Expand Down
5 changes: 3 additions & 2 deletions docsrc/backtesting.polygon.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ Polygon.io Backtesting

**NOTE: Please ensure you have installed the latest lumibot version using ``pip install lumibot --upgrade`` before proceeding as there have been some major changes to the backtesting module in the latest version.**

**You can get an API key at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 Please use the full link to give us credit for the sale, it helps support this project. You can use the coupon code 'LUMI10' for 10% off.**

Polygon.io backtester allows for flexible and robust backtesting. It uses the polygon.io API to fetch pricing data for stocks, options, forex, and cryptocurrencies. This backtester simplifies the process of getting pricing data; simply use the PolygonDataSource and it will automatically fetch pricing data when you call `get_last_price()` or `get_historical_prices()`.

As of this writing, polygon provides up to 2 years of historical data for free. If you pay for an API you can get many years of data and the backtesting will download data much faster because it won't be rate limited.
Expand Down Expand Up @@ -56,7 +58,6 @@ Finally, run the backtest:
datetime_start=backtesting_start,
datetime_end=backtesting_end,
api_key="YOUR_POLYGON_API_KEY",
has_paid_subscription=False, # Set this to True if you have a paid subscription to polygon.io (False assumes you are using the free tier)
)
broker = BacktestingBroker(data_source)
my_strat = MyStrategy(
Expand Down Expand Up @@ -107,7 +108,6 @@ Here's the full code:
datetime_start=backtesting_start,
datetime_end=backtesting_end,
api_key="YOUR_API_KEY_HERE",
has_paid_subscription=False, # Set this to True if you have a paid subscription to polygon.io (False assumes you are using the free tier)
)
broker = BacktestingBroker(data_source)
my_strat = MyStrategy(
Expand All @@ -117,5 +117,6 @@ Here's the full code:
trader.add_strategy(my_strat)
trader.run_all()

**You can get an API key at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 Please use the full link to give us credit for the sale, it helps support this project. You can use the coupon code 'LUMI10' for 10% off.**

In summary, the polygon.io backtester is a powerful tool for fetching pricing data for backtesting various strategies. With its capability to cache data for faster subsequent backtesting and its easy integration with polygon.io API, it is a versatile choice for any backtesting needs.
56 changes: 18 additions & 38 deletions lumibot/backtesting/polygon_backtesting.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

from polygon.exceptions import BadResponse
from termcolor import colored
from urllib3.exceptions import MaxRetryError

from lumibot.data_sources import PandasData
from lumibot.entities import Asset, Data
Expand All @@ -30,16 +29,14 @@ def __init__(
datetime_end,
pandas_data=None,
api_key=None,
has_paid_subscription=False,
**kwargs,
):
super().__init__(
datetime_start=datetime_start, datetime_end=datetime_end, pandas_data=pandas_data, api_key=api_key, **kwargs
)
self.has_paid_subscription = has_paid_subscription

# RESTClient API for Polygon.io polygon-api-client
self.polygon_client = PolygonClient.create(api_key=api_key, paid=has_paid_subscription)
self.polygon_client = PolygonClient.create(api_key=api_key)

@staticmethod
def _enforce_storage_limit(pandas_data: OrderedDict):
Expand Down Expand Up @@ -135,54 +132,37 @@ def _update_pandas_data(self, asset, quote, length, timestep, start_dt=None):
self.datetime_end,
timespan=ts_unit,
quote_asset=quote_asset,
has_paid_subscription=self.has_paid_subscription,
)
except BadResponse as e:
# Assuming e.message or similar attribute contains the error message
formatted_start_datetime = start_datetime.strftime("%Y-%m-%d")
formatted_end_datetime = self.datetime_end.strftime("%Y-%m-%d")
if "Your plan doesn't include this data timeframe" in str(e):
error_message = colored(
"Polygon Access Denied: Your current plan does not support the requested data timeframe "
"Polygon Access Denied: Your subscription does not allow you to backtest that far back in time. "
f"You requested data for {asset_separated} {ts_unit} bars "
f"from {formatted_start_datetime} to {formatted_end_datetime}. "
"Please consider either changing your backtesting timeframe to start later since your "
"subscription does not allow you to backtest that far back, or upgrade your subscription "
"so that you can backtest further back in time. Generally speaking, the more you pay for "
"your subscription, the further back in time you can backtest and the faster you can get "
"data. "
"You can upgrade your Polygon subscription at https://polygon.io/pricing ",
"subscription does not allow you to backtest that far back or upgrade your Polygon "
"subscription."
"You can upgrade your Polygon subscription at at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 "
"Please use the full link to give us credit for the sale, it helps support this project. "
"You can use the coupon code 'LUMI10' for 10% off. ",
color="red")
logging.error(error_message)
# Optionally, inform the user through the application's UI or a notification system
# For CLI or logs, re-raise the exception with a clearer message
raise #Exception("Polygon Access Denied: Upgrade required for requested data timeframe.") from e
raise Exception(error_message) from e
elif "Unknown API Key" in str(e):
error_message = colored(
"Polygon Access Denied: Your API key is invalid. "
"Please check your API key and try again. "
"You can get an API key at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 "
"Please use the full link to give us credit for the sale, it helps support this project. "
"You can use the coupon code 'LUMI10' for 10% off. ",
color="red")
raise Exception(error_message) from e
else:
# Handle other BadResponse exceptions not related to plan limitations
logging.error(traceback.format_exc())
raise
except MaxRetryError as e:
# TODO: Make this just sleep for a bit and retry (there's no need for people to set
# polygon_has_paid_subscription to False)

# Handle MaxRetriesError
error_message = colored(
"Polygon Max Retries Error: The maximum number of retries has been reached. "
"This is probably because you do not have a paid subscription to Polygon. "
"The free version of Polygon has a limit on the number of requests you can make "
"per minute. If you are using the free version of Polygon, you should set "
"polygon_has_paid_subscription to False when you run the backtest() function. eg. \n"
"result = OptionsButterflyCondor.backtest( \n"
" PolygonDataBacktesting, \n"
" backtesting_start, \n"
" backtesting_end, \n"
" polygon_api_key=polygon_api_key, \n"
" polygon_has_paid_subscription=False, # Make sure this is False! \n"
" ) \n"
"Otherwise, you should consider upgrading your subscription to Polygon to avoid this error. "
"You can upgrade your Polygon subscription at https://polygon.io/pricing",
color="red")
logging.error(error_message)
raise
except Exception as e:
# Handle all other exceptions
logging.error(traceback.format_exc())
Expand Down
1 change: 0 additions & 1 deletion lumibot/example_strategies/options_hold_to_expiry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,5 +91,4 @@ def on_trading_iteration(self):
backtesting_end,
benchmark_asset="SPY",
polygon_api_key="YOUR_POLYGON_API_KEY_HERE", # Add your polygon API key here
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category Security Severity Major

Security issue identified: The Polygon API key is hardcoded directly in the code as 'YOUR_POLYGON_API_KEY_HERE'. Hardcoding sensitive information like API keys is a security risk. If an attacker gains access to the codebase, they could extract the API key. To resolve this, remove the API key from the code and instead read it from an environment variable or a separate configuration file that is not committed to version control.

polygon_has_paid_subscription=False,
)
25 changes: 14 additions & 11 deletions lumibot/strategies/_strategy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import datetime
import logging
import warnings
from termcolor import colored
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category Logging

The use of 'termcolor' for coloring log messages is not recommended as it may cause compatibility issues across different environments. Consider using a logging configuration that supports colored output in a more controlled way, such as 'colorlog' or custom log formatters.

from asyncio.log import logger
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category Readability and Maintainability

The import 'from asyncio.log import logger' appears to be unused and should be removed to clean up the code.

from decimal import Decimal
import os
Expand Down Expand Up @@ -773,7 +773,7 @@ def run_backtest(
sell_trading_fees=[],
api_key=None,
polygon_api_key=None,
polygon_has_paid_subscription=False,
polygon_has_paid_subscription=None, # Depricated, this is now automatic. Remove in future versions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category Naming

There is a spelling mistake in the comment next to the polygon_has_paid_subscription parameter. It should be corrected from 'Depricated' to 'Deprecated' to maintain professionalism in the codebase.

indicators_file=None,
show_indicators=True,
save_logfile=False,
Expand Down Expand Up @@ -847,9 +847,6 @@ def run_backtest(
polygon_api_key: str
The polygon api key to use for polygon data. Only required if you are using PolygonDataBacktesting as
the datasource_class. Deprecated, please use 'api_key' instead.
polygon_has_paid_subscription : bool
Whether you have a paid subscription to Polygon. Only required if you are using
PolygonDataBacktesting as the datasource_class.
indicators_file : str
The file to write the indicators to.
show_indicators : bool
Expand Down Expand Up @@ -951,6 +948,17 @@ def run_backtest(
if stats_file is None:
stats_file = f"{logdir}/{basename}_stats.csv"

# Check if polygon_has_paid_subscription is set (it is deprecated and will be removed in the future)
if polygon_has_paid_subscription is not None:
colored_warning = colored("The parameter `polygon_has_paid_subscription` is deprecated and will be removed in the future. "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category Tests

The use of termcolor for logging a deprecation warning within the run_backtest method may not be captured in certain test environments. Consider replacing it with standard logging practices to ensure that the deprecation warning can be captured and asserted in tests.

"This parameter is no longer needed as the PolygonDataBacktesting class will automatically check "
"if you have a paid subscription to Polygon."
"Also, we have partnered with Polygon to provide you a discount on their paid subscription. "
"You can get an API key at https://polygon.io/?utm_source=affiliate&utm_campaign=lumi10 "
"Please use the full link to give us credit for the sale, it helps support this project. "
"You can use the coupon code 'LUMI10' for 10% off your subscription. ", "yellow")
logging.warning(colored_warning)

# #############################################
# Check the data types of the parameters
# #############################################
Expand Down Expand Up @@ -995,8 +1003,6 @@ def run_backtest(
pandas_data=pandas_data,
**kwargs,
)
if hasattr(data_source, "has_paid_subscription"):
data_source.has_paid_subscription = polygon_has_paid_subscription

# if hasattr(data_source, 'pandas_data'):
# data_source.pandas_data = pandas_data
Expand Down Expand Up @@ -1174,7 +1180,7 @@ def backtest(
sell_trading_fees=[],
api_key=None,
polygon_api_key=None,
polygon_has_paid_subscription=False,
polygon_has_paid_subscription=None, # Depricated, this is now automatic. Remove in future versions.
indicators_file=None,
show_indicators=True,
save_logfile=False,
Expand Down Expand Up @@ -1248,9 +1254,6 @@ def backtest(
polygon_api_key : str
The polygon api key to use for polygon data. Only required if you are using PolygonDataBacktesting as
the datasource_class. Depricated, please use 'api_key' instead.
polygon_has_paid_subscription : bool
Whether you have a paid subscription to Polygon. Only required if you are using
PolygonDataBacktesting as the datasource_class.
indicators_file : str
The file to write the indicators to.
show_indicators : bool
Expand Down
Loading
Loading