Skip to content

Commit 3abdee2

Browse files
jonhealy1jonhealy1
authored
Clean recursion (#264)
#204 stac-utils/stac-check#108 - Improved path resolution in recursive validation to handle relative paths more reliably, especially in GitHub repository structures - Fixed issue where validator would get confused by repository structure when validating catalogs with relative paths - Cleaned up path output by removing unnecessary `./` components from displayed paths
1 parent 6b969a5 commit 3abdee2

File tree

1,352 files changed

+80897
-93
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,352 files changed

+80897
-93
lines changed

‎CHANGELOG.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,14 @@ The format is (loosely) based on [Keep a Changelog](http://keepachangelog.com/)
88

99
### Added
1010

11+
## [v3.9.3] - 2025-06-28
12+
1113
### Changed
14+
- Improved path resolution in recursive validation to handle relative paths more reliably, especially in GitHub repository structures [#264](https://github.com/stac-utils/stac-validator/pull/264)
15+
16+
### Fixed
17+
- Fixed issue where validator would get confused by repository structure when validating catalogs with relative paths [#264](https://github.com/stac-utils/stac-validator/pull/264)
18+
- Cleaned up path output by removing unnecessary `./` components from displayed paths [#264](https://github.com/stac-utils/stac-validator/pull/264)
1219

1320
### Removed
1421

@@ -282,7 +289,8 @@ The format is (loosely) based on [Keep a Changelog](http://keepachangelog.com/)
282289
- With the newest version - 1.0.0-beta.2 - items will run through jsonchema validation before the PySTAC validation. The reason for this is that jsonschema will give more informative error messages. This should be addressed better in the future. This is not the case with the --recursive option as time can be a concern here with larger collections.
283290
- Logging. Various additions were made here depending on the options selected. This was done to help assist people to update their STAC collections.
284291

285-
[Unreleased]: https://github.com/sparkgeo/stac-validator/compare/v3.9.2..main
292+
[Unreleased]: https://github.com/sparkgeo/stac-validator/compare/v3.9.3..main
293+
[v3.9.3]: https://github.com/sparkgeo/stac-validator/compare/v3.9.2..v3.9.3
286294
[v3.9.2]: https://github.com/sparkgeo/stac-validator/compare/v3.9.1..v3.9.2
287295
[v3.9.1]: https://github.com/sparkgeo/stac-validator/compare/v3.9.0..v3.9.1
288296
[v3.9.0]: https://github.com/sparkgeo/stac-validator/compare/v3.8.1..v3.9.0

‎setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
from setuptools import setup
44

5-
__version__ = "3.9.2"
5+
__version__ = "3.9.3"
66

77
with open("README.md", "r") as fh:
88
long_description = fh.read()

‎stac_validator/validate.py‎

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -618,12 +618,32 @@ def recursive_validator(self, stac_type: str) -> bool:
618618
if link["rel"] in ("child", "item"):
619619
address = link["href"]
620620
if not is_valid_url(address):
621-
path_parts = str(base_url).split("/")
622-
path_parts.pop(-1)
623-
root = path_parts[0]
624-
for i in range(1, len(path_parts)):
625-
root = f"{root}/{path_parts[i]}"
626-
self.stac_file = f"{root}/{address}"
621+
if is_valid_url(str(base_url)):
622+
# If base is a URL, handle URL joining
623+
from urllib.parse import urljoin
624+
625+
self.stac_file = urljoin(
626+
(
627+
str(base_url) + "/"
628+
if not str(base_url).endswith("/")
629+
else str(base_url)
630+
),
631+
address,
632+
)
633+
else:
634+
# Handle local file paths
635+
current_dir = os.path.dirname(
636+
os.path.abspath(str(base_url))
637+
)
638+
self.stac_file = os.path.normpath(
639+
os.path.join(current_dir, address)
640+
)
641+
# Convert to relative path for cleaner output if it's under the current working directory
642+
try:
643+
self.stac_file = os.path.relpath(self.stac_file)
644+
except ValueError:
645+
# If paths are on different drives (Windows) or other relpath issues, use the absolute path
646+
pass
627647
else:
628648
self.stac_file = address
629649

‎tests/test_data/local_cat/open-science-catalog-testing/catalog.json‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"type": "Catalog",
3+
"id": "osc",
4+
"stac_version": "1.0.0",
5+
"description": "A catalog of publicly available geoscience products, datasets and resources developed in the frame of scientific research Projects funded by ESA EO (Earth Observation)",
6+
"links": [
7+
{
8+
"rel": "root",
9+
"href": "./catalog.json",
10+
"type": "application/json",
11+
"title": "Open Science Catalog"
12+
},
13+
{
14+
"rel": "child",
15+
"href": "./projects/catalog.json",
16+
"type": "application/json",
17+
"title": "Projects"
18+
},
19+
{
20+
"rel": "child",
21+
"href": "./themes/catalog.json",
22+
"type": "application/json",
23+
"title": "Themes"
24+
},
25+
{
26+
"rel": "child",
27+
"href": "./variables/catalog.json",
28+
"type": "application/json",
29+
"title": "Variables"
30+
},
31+
{
32+
"rel": "child",
33+
"href": "./processes/catalog.json",
34+
"type": "application/json",
35+
"title": "Processes"
36+
},
37+
{
38+
"rel": "child",
39+
"href": "./eo-missions/catalog.json",
40+
"type": "application/json",
41+
"title": "EO Missions"
42+
},
43+
{
44+
"rel": "child",
45+
"href": "./products/catalog.json",
46+
"type": "application/json",
47+
"title": "Products"
48+
}
49+
],
50+
"title": "Open Science Catalog"
51+
}

‎tests/test_data/local_cat/open-science-catalog-testing/eo-missions/aeolus/catalog.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"type": "Catalog",
3+
"id": "aeolus",
4+
"stac_version": "1.0.0",
5+
"description": "Aeolus or ADM - Atmospheric Dynamics Mission was an ESA (European Space Agency) Earth Explorer Core Mission -a science-oriented mission within its Living Planet Program. The primary objective was to provide wind profile measurements for an improved analysis of the global three-dimensional wind field. The aim of the mission was to provide global observations of wind profiles with a vertical resolution that could satisfy the accuracy requirements of WMO (World Meteorological Organization). Such knowledge was crucial to the understanding of the atmospheric dynamics, including the global transport of energy, water, aerosols, chemicals and other airborne materials - to be able to deal with many aspects of climate research and climate and weather prediction. ADM-Aeolus represented a demonstration project for the Global Climate Observing System (GCOS).Atmospheric Dynamics Mission (Earth Explorer Core Mission).",
6+
"links": [
7+
{
8+
"rel": "root",
9+
"href": "../../catalog.json",
10+
"type": "application/json",
11+
"title": "Open Science Catalog"
12+
},
13+
{
14+
"rel": "via",
15+
"href": "https://database.eohandbook.com/database/missionsummary.aspx?missionID=377",
16+
"type": "text/html",
17+
"title": "Description"
18+
},
19+
{
20+
"rel": "parent",
21+
"href": "../catalog.json",
22+
"type": "application/json",
23+
"title": "EO Missions"
24+
}
25+
],
26+
"title": "AEOLUS"
27+
}

‎tests/test_data/local_cat/open-science-catalog-testing/eo-missions/aqua-eos-pm-1/catalog.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"type": "Catalog",
3+
"id": "aqua-eos-pm-1",
4+
"stac_version": "1.0.0",
5+
"description": "Jointly funded by NASA (National Aeronautics and Space Administration, USA), INPE (National Institute for Space Research, Brazil) and JAXA (Japan Aerospace Exploration Agency), Aqua (formerly known as EOS/PM-1) is part of NASA’s international Earth Observing System (EOS), as well as their Earth Science Enterprise (ESE) program. Aqua’s main focus is the multidisciplinary study of the Earth’s water cycle including cloud formation, precipitation, radiative properties, air-sea fluxes of energy, carbon and moisture, and sea ice concentrations and extents. Launched in May 2002 from Vandenberg Air Force Base, California, USA, Aqua had a design life of six years, however remains operational as of July 2022. The HSB instrument failed on the 5th of February 2003.",
6+
"links": [
7+
{
8+
"rel": "root",
9+
"href": "../../catalog.json",
10+
"type": "application/json",
11+
"title": "Open Science Catalog"
12+
},
13+
{
14+
"rel": "via",
15+
"href": "https://database.eohandbook.com/database/missionsummary.aspx?missionID=206",
16+
"type": "text/html",
17+
"title": "Description"
18+
},
19+
{
20+
"rel": "parent",
21+
"href": "../catalog.json",
22+
"type": "application/json",
23+
"title": "EO Missions"
24+
}
25+
],
26+
"title": "AQUA (EOS/PM-1)"
27+
}

‎tests/test_data/local_cat/open-science-catalog-testing/eo-missions/aura/catalog.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"type": "Catalog",
3+
"id": "aura",
4+
"stac_version": "1.0.0",
5+
"description": "Aura (formerly EOS/Chem-1) is a chemistry mission of the National Aeronautics and Space Administration (NASA) with the overall objective of studying the chemistry and dynamics of Earth’s atmosphere from the ground to the mesosphere. The Aura mission will provide global surveys of several atmospheric constituents with the goal of monitoring the complex interactions of atmospheric constituents from both natural and man-made sources that are contributing to global change and affect the creation and depletion of ozone.\nLaunched in July 2004, Aura is the third mission in the Earth Observing Satellite (EOS) series, following on from Terra and Aqua.",
6+
"links": [
7+
{
8+
"rel": "root",
9+
"href": "../../catalog.json",
10+
"type": "application/json",
11+
"title": "Open Science Catalog"
12+
},
13+
{
14+
"rel": "via",
15+
"href": "https://database.eohandbook.com/database/missionsummary.aspx?missionID=207",
16+
"type": "text/html",
17+
"title": "Description"
18+
},
19+
{
20+
"rel": "parent",
21+
"href": "../catalog.json",
22+
"type": "application/json",
23+
"title": "EO Missions"
24+
}
25+
],
26+
"title": "Aura"
27+
}

‎tests/test_data/local_cat/open-science-catalog-testing/eo-missions/calipso/catalog.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"type": "Catalog",
3+
"id": "calipso",
4+
"stac_version": "1.0.0",
5+
"description": "Launched in April 2006, CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations) is a cloud and aerosol observation satellite operated by the National Aeronautics and Space Administration (NASA) and Centre National d’Etudes Spatiales (CNES). Its observations are used to determine the role of clouds and aerosols in regulating the Earth’s climate. 3-year nominal mission life, currently in extended operations. Measurements of aerosol and cloud properties for climate predictions, using a 3 channel lidar and passive instruments in formation with CloudSat for coincident observations of radiative fluxes and atmospheric state.",
6+
"links": [
7+
{
8+
"rel": "root",
9+
"href": "../../catalog.json",
10+
"type": "application/json",
11+
"title": "Open Science Catalog"
12+
},
13+
{
14+
"rel": "via",
15+
"href": "https://database.eohandbook.com/database/missionsummary.aspx?missionID=422",
16+
"type": "text/html",
17+
"title": "Description"
18+
},
19+
{
20+
"rel": "parent",
21+
"href": "../catalog.json",
22+
"type": "application/json",
23+
"title": "EO Missions"
24+
}
25+
],
26+
"title": "CALIPSO"
27+
}

0 commit comments

Comments
 (0)