User data migration from filesystem to s3 primary object storage

[jgizolme]

Hello Pytal,

We are planning to migrate user data from two Nextcloud platforms both in version 24.0.8.2 with your user_migration custom_app (version 1.1.0). The Nextcloud source is a primary filesystem storage, the other Nextcloud destination is on a primary object storage.

For now, we can export user data from the filesystem Nextcloud source (with some cleanup like versions:cleanup and/or type selections) and import data to the destination using occ user:export command line ... with some files errors, but globally it works. Users can see files ...

We have two questions :

1- We noticed that the source user directory could be around 180M but the export zip file (done with occ command line) is about 4,9G.
Note : users share big files (web conference video files around 300~500Mo)
Question : Why is that happening ? Is this extra data volume comes from shared files from other users so the export file takes all that the user can see from his account ?

2- Is there a way to exclude files (those shared from others) with the export or import command and recompose shared files at the other destination ?

The best scenario would be : stop the service - export user definitions and datas only (not shared ones) - export shared files definition - import users, data and share definitions - and then open the new service based the Nextcloud with primary object storage.

Thank in advance for your time and your attention.
Great regards,
Jocelyn GIZOLME.

[Pytal]

1. I would suggest unzipping the export file and analyzing the contents to determine which files are taking of so much space

2. There is no way to do that currently but we removed shares from exports in v2.0.1 due to excessive sizes, see https://github.com/nextcloud/user_migration/releases/tag/v2.0.1

[jgizolme]

Hi Pytal, thanks a lot for the replies !
Note : yes, I opened the export file and I confirm that it contains shared files ... big ones (owned by others) !