[Help] No support sample query SQL such as “select * from tableA“?

[casionone]

Environment

• Spark 2.4.3
• Splien agent 1.1.0

We try to exec some sample sql such as "select * from table A", but can not get the spline log information

we found the select query will follow this logic（queryExecution.analyzed), the withAction method will not be executed

[wajda]

Please share your code sample. Also note that by default the lineage is captured lazily, so you need to save the result somewhere in order to get the lineage. If you want to view lineage of temporary data you can enable it in the configuration:

spline.plugins.za.co.absa.spline.harvester.plugin.embedded.NonPersistentActionsCapturePlugin.enabled=true

See: https://github.com/AbsaOSS/spline-spark-agent?tab=readme-ov-file#spark-features-coverage

[casionone]

Please share your code sample. Also note that by default the lineage is captured lazily, so you need to save the result somewhere in order to get the lineage. If you want to view lineage of temporary data you can enable it in the configuration:

spline.plugins.za.co.absa.spline.harvester.plugin.embedded.NonPersistentActionsCapturePlugin.enabled=true

See: https://github.com/AbsaOSS/spline-spark-agent?tab=readme-ov-file#spark-features-coverage

sql just like 'select * form tablea;'
And I find the support command list

Implemented
CreateDataSourceTableAsSelectCommand (org.apache.spark.sql.execution.command)
CreateHiveTableAsSelectCommand (org.apache.spark.sql.hive.execution)
CreateTableCommand (org.apache.spark.sql.execution.command)
DropTableCommand (org.apache.spark.sql.execution.command)
InsertIntoDataSourceDirCommand (org.apache.spark.sql.execution.command)
InsertIntoHadoopFsRelationCommand (org.apache.spark.sql.execution.datasources)
InsertIntoHiveDirCommand (org.apache.spark.sql.hive.execution)
InsertIntoHiveTable (org.apache.spark.sql.hive.execution)
SaveIntoDataSourceCommand (org.apache.spark.sql.execution.datasources)

the simple select sql is not support ?

[wajda]

Yes, It is supported, but the lineage is only captured when the result is stored somewhere. Or enable NonPersistentActoinsCapturePlugin as I mentioned above.