Export to external storage

[kunwp1]

Task Summary

A user requested a feature to export results directly to Google Drive (or other third-party cloud providers). Right now, results can only be exported as a dataset, and users must download locally to use them outside Texera. We'll need to add a direct export feature.

Priority

P2 – Medium

Task Type

• Code Implementation
• Documentation
• Refactor / Cleanup
• Testing / QA
• DevOps / Deployment

[Sentiaus]

Hi, I'm interested in working on this. Has anyone started working on this already? @chenlica

[chenlica]

@Sentiaus I assigned it to you. Welcome back!

@Yicong-Huang As you are testing a new protocol to assign issues, please let @Sentiaus know once it's decided.

[Yicong-Huang]

Hi @Sentiaus welcome back! I sent an email in the dev email list (dev@texera.apache.org) about the new protocol. Here is a snapshot of it https://app.sparkmailapp.com/web-share/D2Y3XOAcQeZnIlLUvYpTQuHXKoD43xGhS9gFgWUY

If you have not subscribed to the email list yet, I highly suggest you to do so. We make all dev related announcements there. Here is the instructions: https://httpd.apache.org/userslist.html

[chenlica]

@xuang7 @aicam @eugenegujing : please have a discussion about this issue, as it appears in use cases today, co-incidentally.

[eugenegujing]

Hi @xuang7 and @aicam Could we have a small discussion after next Tuesday's Group meeting?

[xuang7]

Yes, there is a related use case in the opposite direction. Some users are interested in importing or fetching data from external cloud sources, such as Box or Dropbox, into Texera.

I think both directions may share similar infrastructure, including cloud-provider authentication, connector abstraction, and file transfer APIs. It would be great if we could consider supporting both import and export, since this could benefit more users who already store their data in external cloud services and may not want to manually transfer large datasets.

[chenlica]

Let's give this issue a high priority and solve it in both directions.

[Sentiaus]

@xuang7 @chenlica sorry for the delay, I've linked a PR to close the case for exporting to Google Drive. Importing from Google Drive has not been included, however, I believe it can easily be added by building on top of my changes.

[chenlica]

Thanks @Sentiaus .

Given the increasing importance of this feature, we need a contributor to work on it soon.

Never mind: just saw #5145.

[Yicong-Huang]

I think @Sentiaus already raised a PR #5145.

thanks btw!