Refactor Exec Parsers - remove individual parser classes (#1396)

* Refactor Exec Parsers - unify the code calling similar methods This PR is to remove redundant code in many ExecParsers. We have several ExecParsers with the same functionality in parse code. In this PR, following changes are done: 1. If the Execs don't have expressions, then execName is assigned and ExecInfo object is created from GenericExecParser. 2. If the Execs have expressions then reflection is used to assign the appropriate parser to be used. 3. If there is any additional calculation in the parse function, then only that part is overriden in the ExecParser and the rest of the function uses the GenereicExecParser base code 4.Removed all the files which are not needed anymore Signed-off-by: Niranjan Artal <[email protected]> * Remove reflection instead use case classes to match and call respective Parsers Signed-off-by: Niranjan Artal <[email protected]> * addressed few review comments 1. default Exec name 2. added argument to GenericParser constructor * addressed review comments * Update tests --------- Signed-off-by: Niranjan Artal <[email protected]>
NVIDIA · Nov 6, 2024 · 019ede2 · 019ede2
1 parent 8eaeaa9
commit 019ede2
Show file tree

Hide file tree

Showing 29 changed files with 155 additions and 1,115 deletions.
diff --git a/.../src/main/scala/com/nvidia/spark/rapids/tool/planparser/AggregateInPandasExecParser.scala b/.../src/main/scala/com/nvidia/spark/rapids/tool/planparser/AggregateInPandasExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/ArrowEvalPythonExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/ArrowEvalPythonExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CartesianProductExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CartesianProductExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CoalesceExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CoalesceExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CollectLimitExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/CollectLimitExecParser.scala
diff --git a/...rc/main/scala/com/nvidia/spark/rapids/tool/planparser/CustomShuffleReaderExecParser.scala b/...rc/main/scala/com/nvidia/spark/rapids/tool/planparser/CustomShuffleReaderExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/ExpandExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/ExpandExecParser.scala
diff --git a/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/FilterExecParser.scala b/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/FilterExecParser.scala
diff --git a/.../main/scala/com/nvidia/spark/rapids/tool/planparser/FlatMapGroupsInPandasExecParser.scala b/.../main/scala/com/nvidia/spark/rapids/tool/planparser/FlatMapGroupsInPandasExecParser.scala