Skip to content

Commit

Permalink
Avoid creating a new JSON file for each loop when -p is not specified
Browse files Browse the repository at this point in the history
  • Loading branch information
P-ict0 authored Jun 23, 2024
1 parent b2a1da1 commit 75e974f
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "dynamic-scraper"
version = "1.0.4"
version = "1.0.5"
description = "Dynamic web scraper specifically designed for websites that dynamically load elements (such as AngularJS). It runs at specified time intervals and notifies you about specific updates on the site."
readme = "README.md"
requires-python = ">=3.10"
Expand Down
4 changes: 4 additions & 0 deletions src/web_scraper/utils/json.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,10 @@ def ensure_json(self) -> None:
self.logger.debug(f"(JSON) Removing existing JSON file: {self.path}")
os.remove(self.path)
self.create_empty_json()
self.use_previous = True
self.logger.debug(
f"(JSON) Set project to use the new JSON file: {self.path}"
)
else:
self.logger.debug(f"(JSON) JSON file doesnt exist, creating: {self.path}")
self.create_empty_json()
Expand Down

0 comments on commit 75e974f

Please sign in to comment.