-
Notifications
You must be signed in to change notification settings - Fork 245
chore: Improve expression fallback reporting #2240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2240 +/- ##
============================================
+ Coverage 56.12% 58.51% +2.38%
- Complexity 976 1281 +305
============================================
Files 119 143 +24
Lines 11743 13280 +1537
Branches 2251 2369 +118
============================================
+ Hits 6591 7771 +1180
- Misses 4012 4272 +260
- Partials 1140 1237 +97 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
||
/** We support this feature with full compatibility with Spark */ | ||
case class Compatible(notes: Option[String] = None) extends SupportLevel | ||
|
||
/** We support this feature but results can be different from Spark */ | ||
case class Incompatible(notes: Option[String] = None) extends SupportLevel | ||
|
||
/** We do not support this feature */ | ||
object Unsupported extends SupportLevel |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These move into QueryPlanSerde
now that they are no longer specific to Cast
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, a couple of minor questions that don't require change:
- Should we prefer "Comet" over "we" in comments?
- Do we have an example of what kind of notes would be put in
Compatible
?
Yes, I can update this in the next PR (#2211)
We only have one example of this so far, when casting from String to Date:
So really, when we say "Compatible", we mean Compatible enough that we are willing to allow it by default. |
Thanks for the review @mbutrovich |
Which issue does this PR close?
Part of #2239
Part of #1974
Rationale for this change
We currently have two different approaches to expression fallback reporting.
CometCast
has different support levelsCompatible
,Incompatible
, andUnsupported
which are context sensitive (different types of cast have different support levels)IncompatExpr
This PR is a step towards unifying these approaches.
What changes are included in this PR?
IncompatExpr
traitgetSupportLevel
method toCometExpressionSerde
How are these changes tested?