[SPARK-51691][CORE] SerializationDebugger should swallow exception when try to find the reason of serialization problem

[summaryzb]

What changes were proposed in this pull request?

Catch SparkRuntimeException when deep into serialization exception stack during unit test.

Why are the changes needed?

Present a clearer serialization exception stack hierarchy during test, toString implementation of TreeNode may throw exception since SQLConf.get check. It is helpful for debug the real problem

Does this PR introduce any user-facing change?

Yes, but it only take effect in unit test.
User will see the direct serialization exception and the reference chain beyond the root cause.
Before this pr, user will get confuse when unrelated exception is shown

WARN org.apache.spark.serializer.SerializationDebugger: Exception in serialization debugger
org.apache.spark.SparkRuntimeException: Cannot get SQLConf inside scheduler event loop thread.
    at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotGetSQLConfInSchedulerEventLoopThreadError(QueryExecutionErrors.scala:2002)
    at org.apache.spark.sql.internal.SQLConf$.get(SQLConf.scala:225)
    at org.apache.spark.sql.execution.ScalarSubquery.toString(subquery.scala:69)
    at java.lang.String.valueOf(String.java:2994)
    at scala.collection.mutable.StringBuilder.append(StringBuilder.scala:203)
    at scala.collection.immutable.Stream.addString(Stream.scala:701)
    at scala.collection.TraversableOnce.mkString(TraversableOnce.scala:377)
org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: 
org.apache.spark.SimpleFutureAction

After this pr

org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: org.apache.spark.SimpleFutureAction
Serialization stack:
	- object not serializable (class: org.apache.spark.SimpleFutureAction, value: org.apache.spark.SimpleFutureAction@4050649d)
	- writeObject data (class: java.util.concurrent.ConcurrentHashMap)
	- object (class java.util.concurrent.ConcurrentHashMap)
	....(not shown)

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

[summaryzb]

cc @mridulm

[mridulm]

We can do this even for non testing case, right ? It will be an improvement over earlier behavior (which would have thrown NPE), and minimize side effects of improveException

[summaryzb]

@mridulm Appreciate your time to review
1.NPE have been checked in line#94
2.SPARK-51691 and this follow main resolve the problem, that toString implementation of TreeNode may throw exception since SQLConf.get check during unit test.

Actually non testing case will not encounter similar problem

[mridulm]

NPE I was referring to is what is getting thrown as part of executing toString : what we addressed in #50402

The current reason for NPE might be (2), as you indicated - and so perhaps scoped to Utils.isTesting currently.
But this can happen for any other reason too (including bugs in user udf's for example).

The diff I am proposing is, replace case _: SparkRuntimeException if Utils.isTesting with case _: Exception
I want to make sure this wont cause any other issues.

[summaryzb]

Thanks for detailed explanation
I got your point, how about make it case NonFatal(_)

[LuciferYang]

@summaryzb I reopen SPARK-51691, please rebase and submit a complete pr here, thanks

[summaryzb]

@LuciferYang done, PTAL

[mridulm]

Looks good to me.
+CC @LuciferYang

[LuciferYang]

+1, LGTM

[LuciferYang]

Merged into master. Thanks @summaryzb and @mridulm