You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For example, Unitxt relies on the following fields being part of the instance:
{str} 'recipe_metadata'
{str} 'data_classification_policy'
However, we have 2 operators that delete them from the stream:
SelectFields() which only keep given fields from the stream (in order to only keep the relevant columns in datasets with many).
JoinStreams() which commit these fields to the join operators.
Right now we will add a special handing of these fields in these operators. But this is a more root problem: User can delete fields we are relying on without noticing.
Possible solutions:
The card recipe should be run before any metadata fields are added. This part of the code is the one with the most "editing" of data.
These fields should not be allowed to be edited, unless using a special function, with some sort of mechanism. (And new instances should be forced to add them).
The text was updated successfully, but these errors were encountered:
For example, Unitxt relies on the following fields being part of the instance:
{str} 'recipe_metadata'
{str} 'data_classification_policy'
However, we have 2 operators that delete them from the stream:
Right now we will add a special handing of these fields in these operators. But this is a more root problem: User can delete fields we are relying on without noticing.
Possible solutions:
The text was updated successfully, but these errors were encountered: