Using Simba SQL expressions on Spark 2.1

[gillesgerlo]

Hi guys,

Am I correct in understanding that it's currently not possible to use Simba SQL mode on a Spark 2.1 cluster? I'm much more comfortable using Spark SQL than scala, so would love to hear how you guys are going in migrating this feature.

Thanks!

[dongx-psu]

Yeah, the SQL mode is not available on the standalone version. It is basically because we need to extend the SQL parser to support that and the new antlr based parser is not quite extensible. Anyway, I will try to find another way around when I have time (maybe make a separate parser path in Simba rather than reuse the parser in Spark SQL). Thanks for your interest to our project. Dong On 5/30/2018 8:36:04 AM, Gilles <[email protected]> wrote: Hi guys, Am I correct in understanding that it's currently not possible to use Simba SQL mode on a Spark 2.1 cluster? I'm much more comfortable using Spark SQL than scala, so would love to hear how you guys are going in migrating this feature. Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub [#92], or mute the thread [https://github.com/notifications/unsubscribe-auth/AAu4JgrkIskHGfDVGVFeeG7yOaJoiqrPks5t3rxhgaJpZM4UThr3].

[gillesgerlo]

Thanks, I appreciate that!
I'll start experimenting with the example code you've put up in the mean time. Is any overview available of all functions with their purpose and required parameters?
Cheers,
Gilles

[dongx-psu]

Sorry for late response. I think the best way is to refer the DataFrame or DataSet API. I believe most of them are already in the examples located in the example source folder.