After summarizeWindows flint_df can't able to convert into spark df

[Ashokcs94]

I can able to convert spark df to flint df and apply the summarizeWindows function also working.
After applying any flint function , I can't able to access the df (like i can't convert back to spark df, or can't able to save the result file.

Even flint.show() or spark _df.show() getting error. But function is working correctly, i cant able to access the result df.

[icexelloss]

Hi, Flint DF is also a Spark DF, you should be able to call regular Spark functions on it.

…

On Tue, Feb 12, 2019 at 2:13 AM Ashokcs94 ***@***.***> wrote: I can able to convert spark df to flint df and apply the summarizeWindows function also working. After applying any flint function , I can't able to access the df (like i can't convert back to spark df, or can't able to save the result file. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#62>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAwbrIyU5xPZNcAwb5Cx6jCglXL4d9tOks5vMmmzgaJpZM4a1v6L> .

[Ashokcs94]

But if calls regular spark function on top of flint DF is not working ?
I using (Pyspark) Spark version 2.3 and install the ts-flint (using pip in python3) in spark submit i have added the jars and pyFiles (as flint-0.6.0.jar file). I can able to convert the spark df to flint df (by using flint_df = flintContext.read.dataframe(s_df) ). after I am applied flint function (summarizeWindows) i got the new flint df. With the new flint df i can't able to apply any spark df function (eg.. even in new flint sd flint_df.show() is throwing error.)

Is my Jar issue ? or i missed any installation else ?

I didn't get proper docs for the installation and setup of ts-flint, could you please provide the proper doc in urgent manner.

[icexelloss]

Can you show what exception are you getting

…

On Wed, Feb 13, 2019 at 8:39 AM Ashokcs94 ***@***.***> wrote: But if calls regular spark function on top of flint DF is not working ? I using (Pyspark) Spark version 2.3 and install the ts-flint (using pip in python3) in spark submit i have added the jars and pyFiles (as flint-0.6.0.jar file). I can able to convert the spark df to flint df (by using flint_df = flintContext.read.dataframe(s_df) ). after I am applied flint function (summarizeWindows) i got the new flint df. With the new flint df i can't able to apply any spark df function (eg.. even in new flint sd flint_df.show() is throwing error.) Is my Jar issue ? or anything else ? Please help me out. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAwbrMITMGUkNQ9F3_p0wKxXbbxu__-eks5vNBV-gaJpZM4a1v6L> .

[Ashokcs94]

save_to_parquet_error.txt.txt
flint_df_show_error.txt.txt

I have attached 2 type of error i am getting. (1 - flint_df.write.parquet error , 2 - flint_df.show() error).

[icexelloss]

Looks like your executors don't have flint jar in the class path