You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If we decide to keep the non-matches it's possible to get NaN values in our crossmatch dataframe. For every point in the left partitions we will have a row with the left point information and the information of the respective match on the right (which being inexistent will be set to NaN).
When assigning a row with NaN values on a dataframe, Pandas seems to automatically cast the whole column type to "float". Columns such as Norder_{}_xmatch, Dir_{}_xmatch and Npix_{}_xmatch, therefore have an incorrect type.
We should create an end-to-end test to verify that the column data types of the original catalogs remain unchanged.
Before submitting
Please check the following:
I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a descriprion of what I expected instead.
If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
The text was updated successfully, but these errors were encountered:
@delucchi-cmu yes, supporting None values by default using pyarrow should fix the column types. We're holding off on the merge of #271 this week but I might try to build some end-to-end tests in the meantime to make sure the output columns of the crossmatch indeed remain the same!
Bug report
If we decide to keep the non-matches it's possible to get NaN values in our crossmatch dataframe. For every point in the left partitions we will have a row with the left point information and the information of the respective match on the right (which being inexistent will be set to NaN).
When assigning a row with NaN values on a dataframe, Pandas seems to automatically cast the whole column type to "float". Columns such as
Norder_{}_xmatch
,Dir_{}_xmatch
andNpix_{}_xmatch
, therefore have an incorrect type.We should create an end-to-end test to verify that the column data types of the original catalogs remain unchanged.
Before submitting
Please check the following:
The text was updated successfully, but these errors were encountered: