Downloading multiple tiles #1211

RNOUGAREDE63 · 2024-06-14T09:56:41Z

RNOUGAREDE63
Jun 14, 2024

Hello,

I am facing troubles while trying to download a tile serie with EODAG downloader. In fact, I indicate a serie of tile to download such as : ["Tile_1", "Tile_2", "Tile_3"].
The first tile is finely downloaded, for the requested periode and product (S2_MSI_L1C). However, when the downloading of the second one starts, there is an error appearing :

requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://download.dataspace.copernicus.eu/odata/v1/Products(b5edbb06-c4a2-52d3-b85e-1cbe28b783d8)/$value?token=

I understand that this probleme might come from some kind of authentification issue. But as the first tile is perfectly downloaded, it doesn't seem plausible to me. So is it something to change directly inside the http.py file ?

And plus, this is really bizarre because the search api works well, EODAG is able to find all the tiles requested, for the periode or product specified. And you must indicates username/password for that step....

By the way, I work with the newest Eodag version, 2.12.1

Here is the used code to download products :

defL1C_download(L1C_files_to_download, L1C_available_json, L1C_dir, tile_name, attempts, tempo, cop_dataspace_id, cop_dataspace_pwd):

    
    os.environ["EODAG__COP_DATASPACE__DOWNLOAD__OUTPUTS_PREFIX"] = os.path.join(L1C_dir, tile_name)
    os.environ["EODAG__COP_DATASPACE__AUTH__CREDENTIALS__USERNAME"] = cop_dataspace_id 
    os.environ["EODAG__COP_DATASPACE__AUTH__CREDENTIALS__PASSWORD"] = cop_dataspace_pwd 

    dag = EODataAccessGateway()
    dag.set_preferred_provider("cop_dataspace")

    # We open it in order to get the EOProduct format (necessary to download the product.)
    deserialized_search_results = dag.deserialize(L1C_available_json)

    # This function allows to download the new L1C datas

    plugin_conf = {"extract": True}

    for products in  deserialized_search_results : 
        if products.as_dict()['id'] in L1C_files_to_download : 
            print(f'Trying to download L1C product : '+products.as_dict()['id'])

            for i in range(attempts, 0, -1):
                try:
                  dag.download(products, **plugin_conf)
            
                except :
                    if i == 1:
                        raise
                    print("/! EODAG did not allow you to download the desired product")
                    print("Retry in " + str(tempo) + " seconds ... " + str(i) + " other requests will be done.")
                    time.sleep(int(tempo))

                else:
                    break

I hope I am clear enough to be understood,
thanks in advance for your help

RNOUGAREDE63 · 2024-06-14T14:56:08Z

RNOUGAREDE63
Jun 14, 2024
Author

I forgot to precise : cop_dataspace_id, cop_dataspace_pwd, tiles and start date/end date are well defined in a config file, read at the beginning.

1 reply

sbrunato Jun 14, 2024
Maintainer

can you please post attempts and tempo values, and also the full logs of eodag obtained by executing setup_logging(3) at the beginning of your script (see configuration / logging). This will help us to understand what is happening

RNOUGAREDE63 · 2024-06-17T08:34:18Z

RNOUGAREDE63
Jun 17, 2024
Author

Hello,

Yes sure :
attemps = 2 ;
tempo= 0

and here is the log of eodag obtained by executing my code with setup logging :

 rnougarede@SRVL-PROCESS01:/mnt/tmp/NRTF2D/images-preprocessing$ python test_file.py
/home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/pyproj/__init__.py:89: UserWarning: pyproj unable to set database path.
  _pyproj_global_context_initialize()

 INFO || Parsing the user options...

/mnt/tmp/NRTF2D/images-preprocessing/config/config.cfg

 INFO || The Pre-Processing chain is launched on the 20220101-20220301 period.

 >    S E N T I N E L   2   :  P R E - P R O C E S S I N G

 PROCESS || Checking for new acquisitions : asking for availables L1C (Tile : 33MVT, Period : 20220101 - 20220301)

2024-06-17 10:31:56,532 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:31:57,261 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:31:57,280 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,281 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,281 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,281 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,281 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,281 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:57,285 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:31:57,295 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml
2024-06-17 10:31:57,547 eodag.core                       [INFO    ] (tid=140367515813696) Searching product type 'S2_MSI_L1C' on provider: cop_dataspace
2024-06-17 10:31:57,548 eodag.core                       [DEBUG   ] (tid=140367515813696) Searching for all the products with provider cop_dataspace and a maximum of 1000 items per page.
2024-06-17 10:31:57,548 eodag.core                       [INFO    ] (tid=140367515813696) Iterate search over multiple pages: page #1
2024-06-17 10:31:57,549 eodag.search.base                [DEBUG   ] (tid=140367515813696) Mapping eodag product type to provider product type
2024-06-17 10:31:57,549 eodag.search.base                [DEBUG   ] (tid=140367515813696) Getting provider product type definition parameters for S2_MSI_L1C
2024-06-17 10:31:57,549 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Building the query string that will be used for search
2024-06-17 10:31:57,549 eodag.product.metadata_mapping   [DEBUG   ] (tid=140367515813696) Retrieving queryable metadata from metadata_mapping
2024-06-17 10:31:57,550 eodag.search.qssearch            [INFO    ] (tid=140367515813696) Sending search request: http://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productIdentifier=33MVT&cloudCover=[0,80]&startDate=2022-01-01&completionDate=2022-03-01&productType=S2MSI1C&maxRecords=1000&page=1&exactCount=1
2024-06-17 10:31:58,053 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Adapting 1 plugin results to eodag product representation
2024-06-17 10:31:58,059 eodag.core                       [DEBUG   ] (tid=140367515813696) Iterate over pages: last products found on page 1
2024-06-17 10:31:58,059 eodag.core                       [INFO    ] (tid=140367515813696) Found 1 result(s) on provider 'cop_dataspace'
['S2A_MSIL1C_20220120T091311_N0301_R050_T33MVT_20220120T125659']
/mnt/raster/SENSOR_DATA/S2/L1C/33MVT
[]
2024-06-17 10:31:58,080 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:31:58,285 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,307 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,308 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:31:58,312 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml

 PROCESS || Checking for new acquisitions : asking for availables L1C (Tile : 33MVU, Period : 20220101 - 20220301)

2024-06-17 10:31:58,327 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:31:58,509 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:58,531 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:31:58,535 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml
2024-06-17 10:31:58,538 eodag.core                       [INFO    ] (tid=140367515813696) Searching product type 'S2_MSI_L1C' on provider: cop_dataspace
2024-06-17 10:31:58,539 eodag.core                       [DEBUG   ] (tid=140367515813696) Searching for all the products with provider cop_dataspace and a maximum of 1000 items per page.
2024-06-17 10:31:58,539 eodag.core                       [INFO    ] (tid=140367515813696) Iterate search over multiple pages: page #1
2024-06-17 10:31:58,539 eodag.search.base                [DEBUG   ] (tid=140367515813696) Mapping eodag product type to provider product type
2024-06-17 10:31:58,539 eodag.search.base                [DEBUG   ] (tid=140367515813696) Getting provider product type definition parameters for S2_MSI_L1C
2024-06-17 10:31:58,539 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Building the query string that will be used for search
2024-06-17 10:31:58,539 eodag.product.metadata_mapping   [DEBUG   ] (tid=140367515813696) Retrieving queryable metadata from metadata_mapping
2024-06-17 10:31:58,540 eodag.search.qssearch            [INFO    ] (tid=140367515813696) Sending search request: http://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productIdentifier=33MVU&cloudCover=[0,80]&startDate=2022-01-01&completionDate=2022-03-01&productType=S2MSI1C&maxRecords=1000&page=1&exactCount=1
2024-06-17 10:31:59,061 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Adapting 3 plugin results to eodag product representation
2024-06-17 10:31:59,073 eodag.core                       [DEBUG   ] (tid=140367515813696) Iterate over pages: last products found on page 1
2024-06-17 10:31:59,073 eodag.core                       [INFO    ] (tid=140367515813696) Found 3 result(s) on provider 'cop_dataspace'
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVU_20220105T113715']
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVU_20220105T113715', 'S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833']
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVU_20220105T113715', 'S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833', 'S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612']
/mnt/raster/SENSOR_DATA/S2/L1C/33MVU
[]
2024-06-17 10:31:59,091 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:31:59,291 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:31:59,314 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:31:59,318 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml
Trying to download L1C product : S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833
2024-06-17 10:31:59,320 eodag.auth.keycloak              [DEBUG   ] (tid=140367515813696) fetching new access token
                                                                                 2024-06-17 10:31:59,741 eodag.download.base              [INFO    ] (tid=140367515813696) Download url: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(dbc16517-a474-5f5d-a70b-174560263b50)/$value
                                                                                                                                                                  2024-06-17 10:34:50,010 eodag.download.http              [DEBUG   ] (tid=140367515813696) Download recorded in /mnt/raster/SENSOR_DATA/S2/L1C/33MVU/.downloaded/bfb1c955028cbed8acd8548e36b6fd6e
                                                                                                                                                                  2024-06-17 10:34:50,071 eodag.download.base              [INFO    ] (tid=140367515813696) Extraction activated█| 693961516/693961516 [02:49<00:00, 3161398.64file/s]
                                                                                                                                                                  2024-06-17 10:34:59,520 eodag.download.base              [INFO    ] (tid=140367515813696) Deleting archive S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833.zip
Extracting files from S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833.zip: 100%|████████████████████████████████████| 78/78 [00:09<00:00,  8.21file/s]
2024-06-17 10:34:59,579 eodag.product                    [DEBUG   ] (tid=140367515813696) Product location updated from 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products(dbc16517-a474-5f5d-a70b-174560263b50)/$value' to 'file:///mnt/raster/SENSOR_DATA/S2/L1C/33MVU/S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833/S2B_MSIL1C_20220214T091019_N0400_R050_T33MVU_20220214T112833.SAFE'
2024-06-17 10:34:59,579 eodag.product                    [INFO    ] (tid=140367515813696) Remote location of the product is still available through its 'remote_location' property: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(dbc16517-a474-5f5d-a70b-174560263b50)/$value
Trying to download L1C product : S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612
2024-06-17 10:34:59,581 eodag.auth.keycloak              [DEBUG   ] (tid=140367515813696) using already retrieved access token
                                                                                 2024-06-17 10:34:59,581 eodag.download.base              [INFO    ] (tid=140367515813696) Download url: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(124f76c9-dac9-50b7-98ad-be65395168c7)/$value
                                                                                                                                                                  2024-06-17 10:37:12,286 eodag.download.http              [DEBUG   ] (tid=140367515813696) Download recorded in /mnt/raster/SENSOR_DATA/S2/L1C/33MVU/.downloaded/3de4d7b7bf2c61edbf867b9a2df54c28
                                                                                                                                                                  2024-06-17 10:37:12,355 eodag.download.base              [INFO    ] (tid=140367515813696) Extraction activated| 699912206/699912206 [02:11<00:00, 12150724.53file/s]
                                                                                                                                                                  2024-06-17 10:37:18,414 eodag.download.base              [INFO    ] (tid=140367515813696) Deleting archive S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612.zip
Extracting files from S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612.zip: 100%|████████████████████████████████████| 78/78 [00:06<00:00, 12.77file/s]
2024-06-17 10:37:18,467 eodag.product                    [DEBUG   ] (tid=140367515813696) Product location updated from 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products(124f76c9-dac9-50b7-98ad-be65395168c7)/$value' to 'file:///mnt/raster/SENSOR_DATA/S2/L1C/33MVU/S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612/S2A_MSIL1C_20220209T091131_N0400_R050_T33MVU_20220209T130612.SAFE'
2024-06-17 10:37:18,467 eodag.product                    [INFO    ] (tid=140367515813696) Remote location of the product is still available through its 'remote_location' property: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(124f76c9-dac9-50b7-98ad-be65395168c7)/$value

 PROCESS || Checking for new acquisitions : asking for availables L1C (Tile : 33MVV, Period : 20220101 - 20220301)

2024-06-17 10:37:18,495 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:37:18,678 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,699 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:18,700 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:37:18,704 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml
2024-06-17 10:37:18,707 eodag.core                       [INFO    ] (tid=140367515813696) Searching product type 'S2_MSI_L1C' on provider: cop_dataspace
2024-06-17 10:37:18,707 eodag.core                       [DEBUG   ] (tid=140367515813696) Searching for all the products with provider cop_dataspace and a maximum of 1000 items per page.
2024-06-17 10:37:18,707 eodag.core                       [INFO    ] (tid=140367515813696) Iterate search over multiple pages: page #1
2024-06-17 10:37:18,708 eodag.search.base                [DEBUG   ] (tid=140367515813696) Mapping eodag product type to provider product type
2024-06-17 10:37:18,708 eodag.search.base                [DEBUG   ] (tid=140367515813696) Getting provider product type definition parameters for S2_MSI_L1C
2024-06-17 10:37:18,708 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Building the query string that will be used for search
2024-06-17 10:37:18,708 eodag.product.metadata_mapping   [DEBUG   ] (tid=140367515813696) Retrieving queryable metadata from metadata_mapping
2024-06-17 10:37:18,708 eodag.search.qssearch            [INFO    ] (tid=140367515813696) Sending search request: http://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productIdentifier=33MVV&cloudCover=[0,80]&startDate=2022-01-01&completionDate=2022-03-01&productType=S2MSI1C&maxRecords=1000&page=1&exactCount=1
2024-06-17 10:37:20,191 eodag.search.qssearch            [DEBUG   ] (tid=140367515813696) Adapting 4 plugin results to eodag product representation
2024-06-17 10:37:20,208 eodag.core                       [DEBUG   ] (tid=140367515813696) Iterate over pages: last products found on page 1
2024-06-17 10:37:20,208 eodag.core                       [INFO    ] (tid=140367515813696) Found 4 result(s) on provider 'cop_dataspace'
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVV_20220105T113715']
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVV_20220105T113715', 'S2A_MSIL1C_20220120T091311_N0301_R050_T33MVV_20220120T125659']
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVV_20220105T113715', 'S2A_MSIL1C_20220120T091311_N0301_R050_T33MVV_20220120T125659', 'S2B_MSIL1C_20220214T091019_N0400_R050_T33MVV_20220214T112833']
['S2B_MSIL1C_20220105T091259_N0301_R050_T33MVV_20220105T113715', 'S2A_MSIL1C_20220120T091311_N0301_R050_T33MVV_20220120T125659', 'S2B_MSIL1C_20220214T091019_N0400_R050_T33MVV_20220214T112833', 'S2A_MSIL1C_20220209T091131_N0400_R050_T33MVV_20220209T130612']
/mnt/raster/SENSOR_DATA/S2/L1C/33MVV
['.downloaded', 'S2B_MSIL1C_20220105T091259_N0301_R050_T33MVV_20220105T113715', 'S2A_MSIL1C_20220120T091311_N0301_R050_T33MVV_20220120T125659', 'S2B_MSIL1C_20230130T091129_N0509_R050_T33MVV_20230130T110920', 'S2A_MSIL1C_20230306T090831_N0509_R050_T33MVV_20230306T125013', 'S2B_MSIL1C_20230420T090559_N0509_R050_T33MVV_20230420T111145', 'S2A_MSIL1C_20230405T090551_N0509_R050_T33MVV_20230405T125201', 'S2A_MSIL1C_20230425T090551_N0509_R050_T33MVV_20230425T111141', 'S2B_MSIL1C_20230410T090559_N0509_R050_T33MVV_20230410T111036', 'S2B_MSIL1C_20230430T090559_N0509_R050_T33MVV_20230430T111226', 'S2A_MSIL1C_20230415T090551_N0509_R050_T33MVV_20230415T125158']
2024-06-17 10:37:20,229 eodag.config                     [DEBUG   ] (tid=140367515813696) Loading configuration from /home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/resources/providers.yml
2024-06-17 10:37:20,445 eodag.config                     [INFO    ] (tid=140367515813696) Loading user configuration from: /home/rnougarede/.config/eodag/eodag.yml
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) usgs: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) hydroweb_next: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) wekeo: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,467 eodag.core                       [INFO    ] (tid=140367515813696) creodias_s3: provider needing auth for search has been pruned because no crendentials could be found
2024-06-17 10:37:20,468 eodag.core                       [DEBUG   ] (tid=140367515813696) Opening product types index in /home/rnougarede/.config/eodag/.index
2024-06-17 10:37:20,471 eodag.core                       [INFO    ] (tid=140367515813696) Locations configuration loaded from /home/rnougarede/.config/eodag/locations.yml
Trying to download L1C product : S2B_MSIL1C_20220214T091019_N0400_R050_T33MVV_20220214T112833
2024-06-17 10:37:20,474 eodag.auth.keycloak              [DEBUG   ] (tid=140367515813696) using already retrieved access token
                                                                                 2024-06-17 10:37:20,474 eodag.download.base              [INFO    ] (tid=140367515813696) Download url: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(1c6b7724-fb92-5d49-a905-c67937332fcf)/$value
2024-06-17 10:37:20,476 eodag.download.base              [DEBUG   ] (tid=140367515813696) Record file found (/mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/ce9cb8435d40e26835c025d2321c2fff) but not the actual file
2024-06-17 10:37:20,476 eodag.download.base              [DEBUG   ] (tid=140367515813696) Removing record file : /mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/ce9cb8435d40e26835c025d2321c2fff
2024-06-17 10:37:20,982 eodag.download.http              [ERROR   ] (tid=140367515813696) Error while getting resource :
Traceback (most recent call last):
  File "/home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/plugins/download/http.py", line 683, in _stream_download
    self.stream.raise_for_status()
  File "/home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://download.dataspace.copernicus.eu/odata/v1/Products(1c6b7724-fb92-5d49-a905-c67937332fcf)/$value?token=

{"detail":"Unauthorized"}
2024-06-17 10:37:20,984 eodag.download.http              [DEBUG   ] (tid=140367515813696) Download recorded in /mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/ce9cb8435d40e26835c025d2321c2fff
2024-06-17 10:37:20,985 eodag.download.http              [WARNING ] (tid=140367515813696) Downloaded product is not a Zip File. Please check its file type before using it
S2B_MSIL1C_20220214T091019_N0400_R050_T33MVV_20220214T112833: 0.00B [00:00, ?B/s]
2024-06-17 10:37:20,999 eodag.product                    [DEBUG   ] (tid=140367515813696) Product location updated from 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products(1c6b7724-fb92-5d49-a905-c67937332fcf)/$value' to 'file:///mnt/raster/SENSOR_DATA/S2/L1C/33MVV/S2B_MSIL1C_20220214T091019_N0400_R050_T33MVV_20220214T112833'
2024-06-17 10:37:21,000 eodag.product                    [INFO    ] (tid=140367515813696) Remote location of the product is still available through its 'remote_location' property: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(1c6b7724-fb92-5d49-a905-c67937332fcf)/$value
Trying to download L1C product : S2A_MSIL1C_20220209T091131_N0400_R050_T33MVV_20220209T130612
2024-06-17 10:37:21,001 eodag.auth.keycloak              [DEBUG   ] (tid=140367515813696) using already retrieved access token
                                                                                 2024-06-17 10:37:21,002 eodag.download.base              [INFO    ] (tid=140367515813696) Download url: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(c38ed2bf-473c-5841-a49b-64e7338fbf41)/$value
2024-06-17 10:37:21,004 eodag.download.base              [DEBUG   ] (tid=140367515813696) Record file found (/mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/9f115efb2b7c590e847dd7c53e612319) but not the actual file
2024-06-17 10:37:21,004 eodag.download.base              [DEBUG   ] (tid=140367515813696) Removing record file : /mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/9f115efb2b7c590e847dd7c53e612319
2024-06-17 10:37:21,429 eodag.download.http              [ERROR   ] (tid=140367515813696) Error while getting resource :
Traceback (most recent call last):
  File "/home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/eodag/plugins/download/http.py", line 683, in _stream_download
    self.stream.raise_for_status()
  File "/home/rnougarede/miniconda3/envs/snornf_env/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://download.dataspace.copernicus.eu/odata/v1/Products(c38ed2bf-473c-5841-a49b-64e7338fbf41)/$value?token=

{"detail":"Unauthorized"}
2024-06-17 10:37:21,431 eodag.download.http              [DEBUG   ] (tid=140367515813696) Download recorded in /mnt/raster/SENSOR_DATA/S2/L1C/33MVV/.downloaded/9f115efb2b7c590e847dd7c53e612319
2024-06-17 10:37:21,431 eodag.download.http              [WARNING ] (tid=140367515813696) Downloaded product is not a Zip File. Please check its file type before using it
S2A_MSIL1C_20220209T091131_N0400_R050_T33MVV_20220209T130612: 0.00B [00:00, ?B/s]
2024-06-17 10:37:21,432 eodag.product                    [DEBUG   ] (tid=140367515813696) Product location updated from 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products(c38ed2bf-473c-5841-a49b-64e7338fbf41)/$value' to 'file:///mnt/raster/SENSOR_DATA/S2/L1C/33MVV/S2A_MSIL1C_20220209T091131_N0400_R050_T33MVV_20220209T130612'
2024-06-17 10:37:21,432 eodag.product                    [INFO    ] (tid=140367515813696) Remote location of the product is still available through its 'remote_location' property: https://catalogue.dataspace.copernicus.eu/odata/v1/Products(c38ed2bf-473c-5841-a49b-64e7338fbf41)/$value

1 reply

RNOUGAREDE63 Jun 17, 2024
Author

hope it was what you asked for.

By the way I managed to process 2 tiles in one row. Those tiles were 29NPH and 29NQH. So maybe 2 tiles is the max Eodag can look for and download ? Or the bug comes out only with specifical tiles. Anyway, I have really no idea...

RNOUGAREDE63 · 2024-06-21T07:45:56Z

RNOUGAREDE63
Jun 21, 2024
Author

Hello,

Did you have to study a solution about my case ? I would be relieved if there was a way to fix that problem, I am quite stuck now and can't really move forward with my project.

Thanks in advance,

2 replies

RNOUGAREDE63 Jun 21, 2024
Author

have time*

sbrunato Jun 21, 2024
Maintainer

This authentication issue (token renewal) seams to be fixed on the develop branch. Please retry using eodag installed from sources or using pip install git+https://github.com/CS-SI/eodag.git@develop

RNOUGAREDE63 · 2024-06-21T16:44:18Z

RNOUGAREDE63
Jun 21, 2024
Author

I have tried this, it seems that the pyproj package is not able to identify the CRS anymore. I suspect a miss match between this eodag plugin and the pyproj package. i will retry with a clear environnement.

0 replies

RNOUGAREDE63 · 2024-06-24T09:55:22Z

RNOUGAREDE63
Jun 24, 2024
Author

Hello @sbrunato,

Finally it worked ! I modified options in my search command : just replaced "tileIdentifier" with "id" and eodag is now able to read and download all my tiles serie. Re install pyproj package was also a good idea.
Everything is fine....until next time.

Thanks a lot for your help !
Romain

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Downloading multiple tiles #1211

{{title}}

Replies: 5 comments 4 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Downloading multiple tiles #1211

RNOUGAREDE63 Jun 14, 2024

Replies: 5 comments · 4 replies

RNOUGAREDE63 Jun 14, 2024 Author

sbrunato Jun 14, 2024 Maintainer

RNOUGAREDE63 Jun 17, 2024 Author

RNOUGAREDE63 Jun 17, 2024 Author

RNOUGAREDE63 Jun 21, 2024 Author

RNOUGAREDE63 Jun 21, 2024 Author

sbrunato Jun 21, 2024 Maintainer

RNOUGAREDE63 Jun 21, 2024 Author

RNOUGAREDE63 Jun 24, 2024 Author

RNOUGAREDE63
Jun 14, 2024

Replies: 5 comments 4 replies

RNOUGAREDE63
Jun 14, 2024
Author

sbrunato Jun 14, 2024
Maintainer

RNOUGAREDE63
Jun 17, 2024
Author

RNOUGAREDE63 Jun 17, 2024
Author

RNOUGAREDE63
Jun 21, 2024
Author

RNOUGAREDE63 Jun 21, 2024
Author

sbrunato Jun 21, 2024
Maintainer

RNOUGAREDE63
Jun 21, 2024
Author

RNOUGAREDE63
Jun 24, 2024
Author