This is a simple Unraid plugin that will let you fine-tune the operation of the mover.
- On scheduled runs of mover
- Only actually move file(s) if the the cache drive is getting full (selectable threshold)
- Optionally don't move if a parity check / rebuild is already in-progress
- Optional ability to completely disable the scheduled runs of mover
- Manually executed runs of mover ("Move Now" button) can either follow the rules for schedules, or always move all files
This new fork merge all pull requests after review from Hugenbd's repo (cosmetics, merge skipfiletypes from shares, 4 changes from Swarles below) and add several feature, as for example automatic age threshold and compatibility with Unraid 7.x, and other stuff coming.
First it checks if it's valid for this script run: there must be a cache disk present and an instance of the script must not already be running.
Next, check each of the top-level directories (shares) on the cache disk. For all share with 'Use Cache' setting set to "prefer" or "yes", we use 'find' to create a filtered file list of that share directory. For all share with 'Use Cache' setting set to "only", we use 'du' or 'zfs list' to get total size of that share directory.
The list is sorted by "Use cache", increasing age, pool, and file inode, giving priority for being on cache to "cache only" shares, then "cache prefer" by moving newest from array to cache and older to array, and finally to "cache yes" share by moving only from cache to array. Please note that if age setting is set to something else than "Auto (smart cache)" this script is actually dumb and do not check for size and free space and rely on your own calculations. Files at the top level of the cache or an array disk (i.e not in a share) are never moved.
The list is then passed to original unraid mover. For each file, if the file is not "in use" by any process (as detected by 'fuser' command), then the file is moved, and upon success, deleted from the source disk. If the file already exists on the target, it is not moved and the sourceis not deleted. All meta-data of moved files/directories is preserved: permissions, ownership, extended attributes, and access/modified timestamps. If an error occurs in copying a file, the partial file, if present, is deleted and the operation continues on to the next file.
-
2025.02.24
- Added log message when no share avalible on cache for ZFS dataset. (masterwishx).
- Added option with Notifications to Unraid GUI for error and success messages.
- Fixed issue with (') symbol in ignore File list. Thanks to JayBriers from forums.
- ATIME option is now added for based on age option.
- Debug Logging option added (To print find command and ignored folders/files)
-
2025.02.18.1752
- Check parent empty ZFS dataset for children empty datasets before destroy it. (masterwishx)
- Shell Check Fixes + speedup
ifchecks. - Better help text cosmetic with Bold and italic + added note for Test Mode and some changes in config page.
- Skip cache pool size calculation when only one share found in cache pool.
- Changed minimum threshold of used Primary (cache) space from 5% to 0%.
-
2025.02.12.1707
- Fix issue for shares with spaces Thanks DToX_ from forums. (masterwishx)
- Add option to clean ZFS datasets (masterwishx)
-
2024.09.05.0222
- Fix find not finding hidden files Thanks to solidno8 from forums. (R3yn4ld)
-
2024.09.05.0115
- Add compatibility with unraid 7.x for share_mover (R3yn4ld)
- Fix "integer expression expected" Thanks to: (RonaldJerez)
- Fix "0: command not found" bugs (R3yn4ld)
-
2024.08.18
-
2024.08.17
-
2024.08.15
- Fix ignore list reserved space double quoting (Thanks silver226 see forum post) (R3yn4ld)
- Better empty folder cleaner (R3yn4ld)
- Rewritten to rmdir parent directory of a moved file if empty (drawbacks: will let multidirectory dirs alive)
- Added option to enable/disable empty folder cleaner added in Settings UI
- UI improvements, settings sorted
- Even Better cache priming (hopefully) (R3yn4ld)
- Rewriten Ignore filelist from file and filetypes filtering functions (major)
- Improve calculating size of filtered files and filetypes
- Update calculation from basic/bc to numfmt. Removed bc option
- Added verification for not breaking hardlinks when an hardlinked file is filtered
- Added testmode to cleaning empty folder function, and a min depth of 2.
- Fix ctime bug (R3yn4ld)
-
2024.08.12
- Repair/optimize cache priming (R3yn4ld).
- Adding check to bc (un)installation routine
- Add bc (un)install option (R3yn4ld)
- Force test mode only on major upgrade, keep on minor (R3yn4ld)
- Better cache priming (R3yn4ld)
-
2024.08.11
- Allow operation without array if multiple pools (R3yn4ld): Fixed fatal error bug
- Fixed error message about mover.pid and softstop file when installing the plugin or booting Unraid
-
2024.08.10
- Better filtering with ctime=no (R3yn4ld)
- Improved Synchronization (R3yn4ld):
- Improve synchronization by looking for files on cache first
- Do not count synchronized files twice (freeing/priming target were half achieved)
- Optimize Filtering File and Decision loops regarding Rebalance and Synchronize
- Moved test mode on top of Mover Tuning Page
- Add check for primary storage not existing (dust config files)
-
2024.08.07
- Fix bug introduced by "Allow operation without array if multiple pools" preventing mover to run if less than 2 pools installed.
- 2024-08-07
- Allow operation without array if multiple pools (R3yn4ld) Unraid 7.0.0.beta2 may be required for this to work (6.x gui might not allow to have pool as Primary and Secondary)
- Add cleanup empty folder function (R3yn4ld): Will delete empty folder if file have been moved.
-
2024.08.06
- Bug fixes (R3yn4ld)
- Resynchronize not working for share below moving threshold.
- Internal mover moving files from Secondary to Primary instead of syncing (you may Resynchronize to correct the effect)
- Added Resynchronize all Primary files to Secondary option (R3yn4ld): Resynchronize all Primary files to Secondary. This will resynchronize the Primary (cached) files on both Primary and Secondary (array) so they are backed up and parity protected. All files will be synchronized again independently of modification time. This can be a long operation. Run-once setting will reset back to No after next run
- Minor bug fixes and improvements (R3yn4ld)
- Bug fixes (R3yn4ld)
-
2024.08.05
- Rebalance shares (R3yn4ld): Enhance previous "Repair Primary" option. Renamed it "Rebalance shares". This will move files from shares to their primary and secondary storage if spread elsewhere. May imply moving older files from Primary->Secondary or Secondary->Primary if allowed (cache:prefer or cache:yes) to free some space.
- Bug fixes (R3yn4ld)
-
2024.08.04
- Unraid 7.0.0 beta2 Secondary storage Compatibility (R3yn4ld) minor enhancements (6.12 mover action naming) and... can now move between pools (tested on 7.0.0-beta2) !
- Fix find not ignoring hidden files (R3yn4ld) (Thanks to helpful-tune3401)
- Fix default Settings handling causing a "Unary operator" bug (R3yn4ld) (thanks to Alturismo)
- Add freeing threshold option (R3yn4ld)
-
2024.08.01
-
2024.07.30
-
2024.07.29
- Automatic Array to Cache
- Complete rewrite of file listing functions (find, decide to move..) (R3yn4ld)
- Fix an issue with inaccurate capacity when raid z1 is used Updated zfs functions getting usage of a pool (Freender)
- Added cache mode "prefer" smart moving in "Automatic age" mode (R3yn4ld)
- Added option to "repair" Cache:Only (moving everything on share to cache) and Cache:No (moving everything on share to array) shares (R3yn4ld)
- Added option to synchronize Cache:Yes and Cache:Prefer shares to array so data are parity protected (R3yn4ld)
- Turbo write mode forcing improvement to not wake spinners if not needed (R3yn4ld)
- UI improvements (R3yn4ld)
-
2024.07.10
- Unraid 7.0.0 compatibility (R3yn4ld): original mover now works with "Move Now button follows plug-in filters" set to off - (R3yn4ld)
-
2024.07.07:
-
2024-06-30:
- Automatic age threshold (R3yn4ld)
- Minor spelling corrections & README (Dphelan)
- Merge share skipfiletypes (Davendsai)(add/merge per share skipfilestype to global skips)
- Update Mover.tuning.page (Squid)
-
2023.12.19 (was not in master branch from hugenbd)
- 4 changes from Swarles (Swarles)
- Change "while read" lines in age_mover to "while IFS= read -r" to fix trailing white spaces (Swarles)
- Fix where sometimes mover would not run to mover.old scrip (Swarles)
- Log if "share.cfg" doesn't exists to help trouble shooting (Swarles)
- Check for ca.mover.tuning.cfg file and additional logging. (Swarles)
- 4 changes from Swarles (Swarles)
You can download and install plugins with Community Apps.
You'll find its settings within Settings - Scheduler.
After installation, the default settings are set so that the plugin will be in test mode. You may check /tmp/ca.mover.tuning/Mover_actions_date.list to see if your settings are correct and if the mover will move/keep/sync the files as expected.
There are several commands that can be launched from terminal or a script: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start To start age mover (the internal moving engine) with the settings you set in the GUI
/usr/local/emhttp/plugins/ca.mover.tuning/age_mover softstop To cleanly exit loops (Filtering, Deciding, Moving) and age_mover. While moving/syncing, the ongoing file transfer is not interrupted and softstop occurs after actual file operation.
/usr/local/emhttp/plugins/ca.mover.tuning/age_mover stop To kill all the process (can lead to unfinished or corrupted file transferts while moving).
See the Mover Tuning_ thread on the Unraid support forum for more details and discussions.
This was originally created by Squid and updated by hugenbd, with contributions by Castcore, Swarles, Dphelan and Davendsai. @2025 Updated by masterwishx.