Tiny DeDupe

[Black6spdZ]

Quick Q, what is the difference between Tiny DeDupe and normal?

[007revad]

• Recent DS models have tiny deduplication enabled in synoinfo.conf in DSM 7.2.1
• High-end models have had regular deduplication since 7.1.1 and 7.2

https://github.com/007revad/Synology_enable_Deduplication/wiki/Tiny-vs-regular-Deduplication

Tiny Deduplication only needs 4GB of memory (and maybe slower).
Deduplication only needs 16GB of memory (and maybe faster).

The script will automatically enable Deduplication if the NAS has 16GB or more of memory. If the NAS has less than 16GB and at least 4GB of memory it enables Deduplication.

The --tiny option was only so people with more 16GB or more can force using tiny dedupe (just in case regular dedupe wasn't working correctly).

These models have the tiny dedupe setting enabled already.
DS423+
DS723+
DS923+
DS1522+
DS1621+
DS1621xs+
DS1821+
DS1823xs+

[Black6spdZ]

but what exactly is it? Just manual dedupe only? pros and cons over normal dedupe? I ask because with my 16GB NAS I'm trying to decide if using the RAM resources for normal dedupe outweighs its normal use in caching.

[Black6spdZ]

ok, pulled this from CLI so guessing tiny is dedupe-light?

:/$ synobtrfsdedupe
usage: synobtrfsdedupe [-l] [-v] []

-l dump log to syslog-ng.
-v verbose mode. Print more information for debug.

Commands:
quit-service Send quit command to dedupe daemon

send-command Send command to dedupe daemon
run-schedule Run dedupe schedule
run-light Run dedupe light

udc Dump UDC json
is-deduped Check if any volumes on this machine have run deduplication before.

index-check Check correctness and consistency between sample index and bucket index
dump-record Dump record with give record address
dump-record-headers Dump all record headers in bucket
dump-sample Dump sampled data with given index
dump-sample-header Dump the header of sample index

The following commands are only for debug:

run-service Run dedupe service

pre-share-rename Share rename pre-hook
pre-share-change Share change pre-hook
post-share-create Share create post-hook
post-lun-create LUN create post-hook
pre-lun-change LUN change pre-hook
pre-volume-delete Volume delete pre-hook

dedupe-new Dedupe new created extent in volume
dedupe-light Light weight dedupe
dedupe-rewrite Dedupe rewrote extent in volume
dedupe-full Run full dedupe
dry-run Execute dry-run
gc Run garbage collection
restore-sample Remove all sampled data and reconstruct them by bucket index

[007revad]

tiny is dedupe-light?

Correct.

[Black6spdZ]

without in-depth knowledge from Synology I guess we would assume they limit the hash table size "affecting dedupe efficiency" and/or disable in-line dedupe for the light/tiny? don't suppose you've done any comparison testing?

[007revad]

I haven't had time to do any comparison testing yet, but do intend to. My DS1821+ with 32GB of memory supports both tiny and regular deduplication. And has some NVMe drives and a SSD that are solely used for testing things. I'll compare deduping the same files with tiny dedupe and regular dedupe.

[007revad]

would the method you used to enable dedupe on other models also work to enable surveillance station "deep analytics" on non DVA model boxes?

What model Synology NAS do you have?

The big difference is surveillance station "deep analytics" is included in the surveillance station package for the DVA models.

• It's 370 to 440MB for the DVA models.
• It's only 170MB for the DS and RS models.

I haven't looked inside the surveillance station packages to see what's different yet.

After comparing DSM 7.2.1 for the DVA1622 and DS720+ there is very little different. I compared the DVA1622 and DS720+ because they both use the same 2-bay hot swap chassis and use the same Celeron J4125 CPU.

The DVA1622 requires 6GB of memory so 6GB is probably the minimum needed.

DVA1622 extra settings in /etc.defaults/synoinfo.conf compared to DS720+

is_dva_model="yes"
support_localinstall="yes"

DVA1622 different settings in /etc.defaults/synoinfo.conf compared to DS720+

These are all "yes" in DS720+

restart_clean_session="no"            # probably important
support_ihm="no"                      # don't know what this is for?
support_kmip="no"                     # no volume encryption
support_printer="no"                  # who cares
support_raid_disk_replacement="no"    # not sure why!?!?
support_ssd_cache="no"                # probably because it has no M.2 slots and only 2 bays
support_usb_printer="no"              # who cares
support_volume_encryption="no"        # no volume encryption
support_worm="no"                     # no immutable snapshots
support_write_cache="no"              # because no ssd cache support
supportraidcross="no"                 # not sure what this is for?

Other differences in synoinfo.conf are only because DVA1622 only has 1 LAN port and no eSATA port.

The DVA1622 has 9 extra files which are only to download and install surveillance station during the first setup.

The DVA3219 and DVA3221 do not have any of those 9 files. But I also didn't find the surveillance station package included in DSM.

[Black6spdZ]

interesting.. I added the missing is_dva_model to my 3622 synoinfo.conf and attempted to install the SS openvino package but it tells me unsupported model.. this package however works properly on a 1622

[007revad]

You have a DVA1622 ?

The DVA1622 has a Geminilake CPU as does my DS720+ so I tried adding is_dva_model="yes" to /etc.defaults/synoinfo.conf and /etc/synoinfo.conf but when I tried to manually install SurveillanceStation-x86_64-9.1.4-10998_openvino.spk I got the following message:

This package is not supported on the platform of Synology NAS or is incompatible with the current DSM version.

So then I changed product="Synology NAS" to product="Synology NVR" in both synoinfo files and I got the following message:

This package is not supported on the platform of Synology NVR or is incompatible with the current DSM version.

I'll unpack the .spk file to see what it is checking.

[007revad]

I unpacked the openvino spk file and the minimum DSM version 7.0 and supported CPU arch is x86_64

I did find what was stopping it installing... and managed to install it on my DS720+. 😃

I had to change unique in both synoinfo.conf files from: unique="synology_geminilake_720+"
to: unique="synology_geminilake_dva1622"

• SurveillanceStation-x86_64-9.1.4-10998_openvino will only install on: unique="synology_geminilake_dva1622"
• SurveillanceStation-x86_64-9.1.4-10998_DVA_3221 will only install on: unique="synology_denverton_dva3221" and needs an NVIDA GPU
• SurveillanceStation-x86_64-9.1.4-10998_DVA_3219 will only install on: unique="synology_denverton_dva3219"

I don't think you need a geminilake or denverton CPU. They should install on any Synology with an x86_64 CPU as long as unique is set to one of the 3 supported models and is_dva_model="yes" is set.

I don't know these surveillance station packages require a GPU or CPU with an iGPU.

It would be interesting to know if it works on your DS3622xs+ which does not an iGPU... but has beefy CPU that should cope ok.

Obviously you would not want to leave your DS3622xs+ set as a DVA3221 as that would probably result in DSM updates downloading and installing the wrong model's DSM.

Once surveillance station is installed there are 3 files with functions that check if the NAS is a DVA model. These 3 files can be edited after surveillance station has installed.

• One of the functions checks if the upnpmodelname in synoinfo.conf is either DVA3219, DVA3221 or DVA1622. We can either edit the upnpmodelnamei n synoinfo.conf or edit the function in the file to add the DS or RS model.
• The other 2 I'm not sure about, yet.

[007revad]

Okay, as soon as I change unique="synology_geminilake_dva1622" back to unique="synology_geminilake_720+" in /etc/sinnoinfo.conf Surveillance Station in Package Center instantly goes from "Running" to "Error". Changing it back dva1622 instantly changes it back to "Running". I'm very close but not quite there yet...

[Black6spdZ]

this is exciting news! I know the 1622 uses the UHD 600 iGPU but may extend to other iGPU models.. at least from Gemini Lake up, the 3219 uses Nvidia 1050+ and 3221 uses 1650+

[007revad]

I took a break today. But did notice a problem. I rebooted the DS720+ and when I tried to open it's DSM webpage it refused to connect.

So I tried with Synology Assistant and noticed 3 things:

1. The server name had changed to SynologyNAS.
2. The model was showing as DS720+ instead of DVA1622.
3. The status was showing as Migratable.

When I clicked on connect I got a Welcome Back message saying "We've detected that you had moved the hard drives from a DVA1622 to a DS720+. You can migrate your data and settings to your NAS now."

[Black6spdZ]

so there must be other config files with the model # / type?