Skip to content

BUG: Heasarch.locate_data() triggers 414 Client Error: Request-URI Too Large #3276

Open
@bsipocz

Description

@bsipocz

While reviewing #3275 I run into the following error. If a code fix is not possible, then we need to include some upper limit, or add some docstring documentation about some reasonable limits for this functionality.

This is both on main and on the PR's branch.

cc @zoghbi-a

>>> from astroquery.heasarc import Heasarc
>>> from astropy.coordinates import SkyCoord
>>> pos = SkyCoord.from_name('Cas A')
>>> tab = Heasarc.query_region(pos, catalog='nicermastr')
>>> Heasarc.locate_data(tab)
---------------------------------------------------------------------------
E19                                       Traceback (most recent call last)
File ~/munka/devel/pyvo/pyvo/dal/query.py:260, in DALQuery.execute_votable(self, post)
    259 try:
--> 260     return votableparse(self.execute_stream(post=post).read)
    261 except Exception as e:

File ~/munka/devel/astropy/astropy/io/votable/table.py:164, in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
    161 with iterparser.get_xml_iterator(
    162     source, _debug_python_based_parser=_debug_python_based_parser
    163 ) as iterator:
--> 164     return tree.VOTableFile(config=config, pos=(1, 1)).parse(iterator, config)

File ~/munka/devel/astropy/astropy/io/votable/tree.py:4282, in VOTableFile.parse(self, iterator, config)
   4281         else:
-> 4282             vo_raise(E19, (), config, pos)
   4283 config.update(self._get_version_checks())

File ~/munka/devel/astropy/astropy/io/votable/exceptions.py:124, in vo_raise(exception_class, args, config, pos)
    123     config = {}
--> 124 raise exception_class(args, config, pos)

E19: None:4:0: E19: File does not appear to be a VOTABLE

During handling of the above exception, another exception occurred:

DALServiceError                           Traceback (most recent call last)
Cell In[1], line 5
      3 pos = SkyCoord.from_name('Cas A')
      4 tab = Heasarc.query_region(pos, catalog='nicermastr')
----> 5 Heasarc.locate_data(tab)

File ~/munka/devel/astroquery/astroquery/heasarc/core.py:526, in HeasarcClass.locate_data(self, query_result, catalog_name)
    519 dlink_url = f'{self.VO_URL}/datalink/{catalog_name}'
    521 query = pyvo.dal.adhoc.DatalinkQuery(
    522     baseurl=dlink_url,
    523     id=query_result['__row'],
    524     session=self._session
    525 )
--> 526 dl_result = query.execute().to_table()
    527 # include rows that have directory links (i.e. data) and those
    528 # that report errors (usually means there are no data products)
    529 dl_result = dl_result[np.ma.mask_or(
    530     dl_result['content_type'] == 'directory',
    531     dl_result['error_message'] != ''
    532 )]

File ~/munka/devel/pyvo/pyvo/dal/adhoc.py:555, in DatalinkQuery.execute(self, post)
    540 def execute(self, post=False):
    541     """
    542     submit the query and return the results as a DatalinkResults instance
    543 
   (...)
    552        for errors parsing the VOTable response
    553     """
    554     return DatalinkResults(
--> 555         self.execute_votable(post=post),
    556         url=self.queryurl,
    557         original_row=self.original_row,
    558         session=self._session)

File ~/munka/devel/pyvo/pyvo/dal/query.py:262, in DALQuery.execute_votable(self, post)
    260     return votableparse(self.execute_stream(post=post).read)
    261 except Exception as e:
--> 262     self.raise_if_error()
    263     raise DALFormatError(e, self.queryurl)

File ~/munka/devel/pyvo/pyvo/dal/query.py:271, in DALQuery.raise_if_error(self)
    269 if self._ex:
    270     e = self._ex
--> 271     raise DALServiceError.from_except(e, self.queryurl)

DALServiceError: 414 Client Error: Request-URI Too Large for url: https://heasarc.gsfc.nasa.gov/xamin/vo/datalink/nicermastr?ID=53845&ID=53846&ID=53847&ID=53848&ID=53854&ID=54680&ID=54682&ID=54683&ID=54684&ID=54685&ID=54686&ID=54687&ID=54688&ID=54689&ID=54690&ID=54691&ID=54692&ID=54693&ID=54694&ID=54695&ID=54696&ID=54697&ID=54698&ID=54699&ID=54700&ID=54701&ID=54702&ID=54703&ID=54704&ID=54705&ID=54706&ID=54707&ID=54708&ID=54709&ID=54710&ID=54711&ID=54712&ID=54713&ID=54714&ID=54715&ID=54716&ID=54717&ID=54718&ID=54719&ID=54720&ID=54721&ID=54722&ID=54723&ID=54724&ID=54725&ID=54726&ID=54727&ID=54728&ID=54729&ID=54730&ID=54731&ID=54732&ID=54733&ID=54734&ID=54735&ID=54736&ID=54737&ID=54738&ID=54739&ID=54740&ID=54741&ID=54742&ID=54743&ID=54744&ID=54745&ID=54746&ID=54747&ID=54748&ID=54749&ID=54750&ID=54751&ID=54752&ID=54753&ID=54754&ID=54755&ID=54756&ID=54757&ID=54758&ID=54759&ID=54760&ID=54761&ID=54762&ID=54763&ID=54764&ID=54765&ID=54766&ID=54767&ID=54768&ID=54769&ID=54770&ID=54771&ID=54772&ID=54773&ID=54774&ID=54775&ID=54776&ID=54777&ID=54778&ID=54779&ID=54780&ID=54781&ID=54782&ID=54783&ID=54784&ID=54785&ID=54786&ID=54787&ID=54788&ID=54789&ID=54790&ID=54791&ID=54792&ID=54793&ID=54794&ID=54795&ID=54796&ID=54797&ID=54798&ID=54799&ID=54800&ID=54801&ID=54802&ID=54803&ID=54804&ID=54805&ID=54806&ID=54807&ID=54808&ID=54809&ID=54810&ID=54811&ID=54812&ID=54813&ID=54814&ID=54815&ID=54816&ID=54817&ID=54818&ID=54819&ID=54820&ID=54821&ID=54822&ID=54823&ID=54824&ID=54825&ID=54826&ID=54827&ID=54828&ID=54829&ID=54830&ID=54831&ID=54832&ID=54833&ID=54834&ID=54835&ID=54836&ID=54837&ID=54838&ID=54839&ID=54840&ID=54841&ID=54842&ID=54843&ID=54844&ID=54845&ID=54846&ID=54847&ID=54848&ID=54849&ID=54850&ID=54851&ID=54852&ID=54853&ID=54854&ID=54855&ID=54856&ID=54857&ID=54858&ID=54859&ID=54860&ID=54861&ID=54862&ID=54863&ID=54864&ID=54865&ID=54866&ID=54867&ID=54868&ID=54869&ID=54870&ID=54871&ID=54872&ID=54873&ID=54874&ID=54875&ID=54876&ID=54877&ID=54878&ID=54879&ID=54880&ID=54881&ID=54882&ID=54883&ID=54884&ID=54885&ID=54886&ID=54887&ID=54888&ID=54889&ID=54890&ID=54891&ID=54892&ID=54893&ID=54894&ID=54895&ID=54896&ID=54897&ID=54898&ID=54899&ID=54900&ID=54901&ID=54902&ID=54903&ID=54904&ID=54905&ID=54906&ID=54907&ID=54908&ID=54909&ID=54910&ID=54911&ID=54912&ID=54913&ID=54914&ID=54915&ID=54916&ID=54917&ID=54918&ID=54919&ID=54920&ID=54921&ID=54922&ID=54923&ID=54924&ID=54925&ID=54926&ID=54927&ID=54928&ID=54929&ID=54930&ID=54931&ID=54932&ID=54933&ID=54934&ID=54935&ID=54936&ID=54937&ID=54938&ID=54939&ID=54940&ID=54941&ID=54942&ID=54943&ID=54944&ID=54945&ID=54946&ID=54947&ID=54948&ID=54949&ID=54950&ID=54951&ID=54952&ID=54953&ID=54954&ID=54955&ID=54956&ID=54957&ID=54958&ID=54959&ID=54960&ID=54961&ID=54962&ID=54963&ID=54964&ID=54965&ID=54966&ID=54967&ID=54968&ID=54969&ID=54970&ID=54971&ID=54972&ID=54973&ID=54974&ID=54975&ID=54976&ID=54977&ID=54978&ID=54979&ID=54980&ID=54981&ID=54982&ID=54983&ID=54984&ID=54985&ID=54986&ID=54987&ID=54988&ID=54989&ID=54990&ID=54991&ID=54992&ID=54993&ID=54994&ID=54995&ID=54996&ID=54997&ID=54998&ID=54999&ID=55000&ID=55001&ID=55002&ID=55003&ID=55004&ID=55005&ID=55006&ID=55007&ID=55008&ID=55009&ID=55010&ID=55011&ID=55012&ID=55013&ID=55014&ID=55015&ID=55016&ID=55017&ID=55018&ID=55019&ID=55020&ID=55021&ID=55022&ID=55023&ID=55024&ID=55025&ID=55026&ID=55027&ID=55028&ID=55029&ID=55030&ID=55031&ID=55032&ID=55033&ID=55034&ID=55035&ID=55036&ID=55037&ID=55038&ID=55039&ID=55040&ID=55041&ID=55042&ID=55043&ID=55044&ID=55045&ID=55046&ID=55047&ID=55048&ID=55049&ID=55050&ID=55051&ID=55052&ID=55053&ID=55054&ID=55055&ID=55056&ID=55057&ID=55058&ID=55059&ID=55060&ID=55061&ID=55062&ID=55063&ID=55064&ID=55065&ID=55066&ID=55067&ID=55068&ID=55069&ID=55070&ID=55071&ID=55072&ID=55073&ID=55074&ID=55075&ID=55076&ID=55077&ID=55078&ID=55079&ID=55080&ID=55081&ID=55082&ID=55083&ID=55084&ID=55085&ID=55086&ID=55087&ID=55088&ID=55089&ID=55090&ID=55091&ID=55092&ID=55093&ID=55094&ID=55095&ID=55096&ID=55097&ID=55098&ID=55099&ID=55100&ID=55101&ID=55102&ID=55103&ID=55104&ID=55105&ID=55106&ID=55107&ID=55108&ID=55109&ID=55110&ID=55111&ID=55112&ID=55113&ID=55114&ID=55115&ID=55116&ID=55117&ID=55118&ID=55119&ID=55120&ID=55121&ID=55122&ID=55123&ID=55124&ID=55125&ID=55126&ID=55127&ID=55128&ID=55129&ID=55130&ID=55131&ID=55132&ID=55133&ID=55134&ID=55135&ID=55136&ID=55137&ID=55138&ID=55139&ID=55140&ID=55141&ID=55142&ID=55143&ID=55144&ID=55145&ID=55146&ID=55147&ID=55148&ID=55149&ID=55150&ID=55151&ID=55152&ID=55153&ID=55154&ID=55155&ID=55156&ID=55157&ID=55158&ID=55159&ID=55160&ID=55161&ID=55162&ID=55163&ID=55164&ID=55165&ID=55166&ID=55167

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions