Import avoiding duplication but filling missing slots

[jwetzig]

Greetings,
I just recently noted, that quite a lot (90%) of my about 50.000 photos are actually missing the "real" file and display only the preview. Call me inattentive, but I had meanwhile updated the metadata heavily and added more photos (for most of which I have a backup "somewhere" but only the "imported on ..." trace).
Now I am left with a swiss cheese photolibrary of 400GB.
Is there a way to partially import from older backup libraries and backup folders? I guess what I am looking for is a rule like "if there is already a entry with same name and/or footprint and metadata present but file missing use that entry's metadata and paste that into the new import, then throw away the old entry". Avoid duplicates (--skip-dup) wouldn't serve, the way I understand the documentation.
Regards
J

[RhetTbull]

This would be possible with a custom import script (not too hard and I'm happy to help) but not "out of the box" with the osxphotos import command. I don't quite understand the situation with your library -- how did it get like this? E.g. was it an iCloud library that was set to "Optimize Mac Storage" and is no no longer an iCloud library? Were these "referenced" files that were not originally copied to the library?

The key to making this work is to match backup file to the current entry in the library. For images, Photos computes a fingerprint (effectively a unique hash) that could be compared assuming the backups are exact copies of what was originally imported. The fingerprint is note stored for videos so those would have to be compared on name + original size for example.

osxphotos cannot delete photos but it could add them to a "Delete these" album once the backup is found and imported and metadata copied over. All metadata with exception of persons can be copied -- Photos does not provide a way to edit the "person in photo" information outside of the GUI.

[RhetTbull]

One other thought: you mentioned backups of the Photos library. If you've never used the "repair photos library" option and the backup libraries have the missing photos, matching will be trivial as the unique ID (UUID) that photos stores in the database should match making matching easy. If you ever repaired the library or created it from scratch by downloading from iCloud, the UUIDs will be different though.