Nextcloud client 3.1 syncs "ignored files"

[libeanim]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Expected behaviour

Nextcloud client should ignore files and folders that have been added to the "Ignored Files" list through the UI.

Actual behaviour

Nextcloud client syncs folders that are in the "ignored files" list since the update to 3.1.
I've experienced this bug on "folder sync-connection"-level (see "Steps to reproduce" if unclear), it is also possible that it happens when using the global ignore file.

Steps to reproduce

1. Add a new 'Folder Sync Connection'
2. Click on the three dots on the settings window next to the newly added "folder sync connection" and select "Edit Ignored Files" from the dropdown menu.
3. Add a folder you want nextcloud to ignore
4. Start sync and create a folder that is in the ignored list
5. Watch nextcloud client sync this folder

Client configuration

Client version: 3.1.0-20201214.180315.7f4ffd0f7-1.0~focal1

Using Qt 5.12.8, built against Qt 5.12.8

Using Qt platform plugin 'xcb'

Using 'OpenSSL 1.1.1f 31 Mar 2020'

Running on Ubuntu 20.04.1 LTS, x86_64

Installation path of client: /usr/bin/nextcloud

Logs

Haven't seen any errors in the logs, it seems like a general client problem. If it is only my installation I can add this information.

[er-vin]

@camilasan could be related to your work in that area? rings any bell?

[nursoda]

Not only applies to per-instance ignore lists but also for global ignore list (in Settings→General).

[benjamindahl]

this slows down my client extremely. How can I go back to 3.0.3 under ubuntu? There's no 3.0.3. package in the repository anymore.

[expert-geeks]

You might find this link helpful, https://download.nextcloud.com/desktop/releases/

[benjamindahl]

You might find this link helpful, https://download.nextcloud.com/desktop/releases/

Thanks a lot. It works for the moment.

[libeanim]

Hmm so the global ignore file seems to work for me and it also appears in the log:

2020-12-21 18:02:28:264 [ info nextcloud.sync.configfile ]:
Adding user defined ignore list to csync: "/home/andreas/.config/Nextcloud/sync-exclude.lst"

I'm just guessing, but @camilasan is it possible that the new path building in the directory traversal function has a bug?

[roundoutandabout]

@libeanim I also have this issue. Nextcloud-3.1.0-x86_64.AppImage syncs everything ignoring the ignore list.
Nextcloud-3.0.3-x86_64.AppImage works fine.
I'm using KDE Neon 5.20

[albgus]

I noticed this with 3.1.0 and 3.1.1 on Windows as well.

[chuck-r]

This isn't just a Windows problem. I noticed the same thing when I upgraded Nextcloud using the Linux AppImage yesterday. Unfortunately, I'm not sure what version I came from, but upgrading to 3.1.1 caused it to try to sync all of my excluded files. I downgraded to 3.0.3 as mentioned here and I still have the same problem.

EDIT: The following workaround/hackery only works for me on 3.0.3, but not 3.1.1

However I noticed that if I move my excluded files out of the sync path and let the folders sync sans the ignored files it syncs fine (of course, since it doesn't have to consider them). Then, if I move the ignored files back into the folder once the initial sync finishes successfully the files are properly ignored -- at least in my very small test case.

In one of my problem folders I had two subfolders, one with 50 or so fairly large source archives within a couple dozen folders. When I moved that folder into place it had no problems and did not synchronize them as the whole folder was on the ignore list. However, I had an issue with moving my second folder into place which had 15,357 folders and 330,359 files -- basically it at up all 119GB of my RAM before I finally had to kill it. But, that's a separate issue I think.

So, there is some disparity between the ignore list when it comes to a full sync (or a force resync, presumably) and an update sync (when it detects a file change).

[qu1ck]

I have this issue with 3.1.1 on windows. Folder specific ignore list doesn't work, global one does.

[chuck-r]

Thanks @qu1ck! I was getting really frustrated by not being able to sync a couple of folders due to this bug. Adding the exception globally did fix the problem, albeit not a perfect solution. So, it seems that the client it's finding/reading/parsing the folder-specific sync excludes. At least that gets us closer to the right direction. I've been investigating this one myself as well, but I'm still trying to get the "lay of the land" as it were with the Nextcloud Client source.

[StarSmasher44]

If I may provide some small insight into my actions dealing with this, I had to exclude log folders & contents, adding just Logs/ did not work.

However, adding both 'Logs/' and 'Logs/*' made the entire folder plus contents be ignored as it should. Not sure if it is useful information, but that worked well enough for me.

[camilasan]

I will investigate it.

[FiC5020]

Have the same problem / issue too. Upgraded from 3.0.3 auf 3.1.1. Now the the client would sync 300 GB off Data which are excluded. Downgrade to 3.0.3 the ignore list works.

Problem solved with an older client.

[StarSmasher44]

Have the same problem / issue too. Upgraded from 3.0.3 auf 3.1.1. Now the the client would sync 300 GB off Data which are excluded. Downgrade to 3.0.3 the ignore list works.

Problem solved with an older client.

Curious, have you tried the method I spoke of a bit above here? it worked out for me in that way, it may also work for you without the need to go back versions :)

[RM-0454]

3. Add a folder you want nextcloud to ignore
4. Start sync and create a folder that is in the ignored list
5. Watch nextcloud client sync this folder

@libeanim I have been having the same issue, but after the following fix worked, I think I know the bug's source.

However I noticed that if I move my excluded files out of the sync path and let the folders sync sans the ignored files it syncs fine (of course, since it doesn't have to consider them). Then, if I move the ignored files back into the folder once the initial sync finishes successfully the files are properly ignored -- at least in my very small test case.

After trying both, my sync excludes are working and I think I know why.

Let's walk through our process again. Pretend we want to exclude the directory foo from a synced folder.

• So under Folder Sync Settings we click "Edit Ignored Files"
• Add foo\ (because we are on windows)
• In file explorer, create new folder and name it "foo"
• Watch in helpless anger as nextcloud proceeds to sync "foo" anyway

However, if instead of creating foo, we copy foo into the synced folder, nextcloud ignores it (as it should). What's the difference?

Theory
When a new folder is created it is automatically assigned the name "New folder" (in windows). "New folder" isn't an excluded file, so it is immediately synced even as you are renaming it. Then when you do rename it, nexctloud records the change in the directory's name without double checking it against the exclude list. That is my suspicion anyway.

OS: Windows 7 | NC Version 3.1.1

[timpur]

I found the opposite of #2728 (comment)

Global Ignore list didn't work but folder ignore list did.

Standard format node_modules/ worked in folder ignore list not root list.

Its like it wasn't reading the list at all, or maybe the working dir is incorrect from root ?

(windows 3.1.2)

[dasaweb]

this slows down my client extremely. How can I go back to 3.0.3 under ubuntu? There's no 3.0.3. package in the repository anymore.

Are you sure that this is a related problem? I see the same 2 problems when I update from 3.0.3 to 3.1.x:

1. .sync-exclude.lst are ignored.
2. The client get's extremly slow.

I'm not sure if these two are related. But I hope, that this is the case, so solving one problem would also solve the other ;-)

[benjamindahl]

this slows down my client extremely. How can I go back to 3.0.3 under ubuntu? There's no 3.0.3. package in the repository anymore.

Are you sure that this is a related problem? I see the same 2 problems when I update from 3.0.3 to 3.1.x:

1. .sync-exclude.lst are ignored.
2. The client get's extremly slow.

I'm not sure if these two are related. But I hope, that this is the case, so solving one problem would also solve the other ;-)

I don't know exactly what the problem is but I'll stick now with 3.0.3 for a while.

[dasaweb]

Staying with version 3.0.3 is the only option for me at the moment. However, with version 3.0.3 I have this additional problem: #324 So no working client for me right now... :-/

[talesam]

This should be simpler, in the GUI to have an option to deselect what you don't want to synchronize, is that so complex to do?

[camilasan]

* The root `sync-exlcude.lst` is not recognized. It is synced but it is not used.

it should be fixed by #2723 and #3882.

[jokay]

@camilasan "should" or "is" fixed? :)

[SMESH109]

I wondered the same.
#3882 is already merged into main and set for Release 3.4
#2723 has been rejected from automatic merge and has to be reviewed manually before merging.

[camilasan]

I was too eager to close the issue :)

[LecrisUT]

I would like to test both of these PRs, but there are no macos artifacts generated, and I'm having trouble compiling from source due to homebrew qt. If anyone can confirm that the root .sync-exclude.lst is picked up locally, and whether or not it's shared among other clients (preferably it should unless .sync-exclude.lst pattern is included in itself). Probably any OS would suffice.

[dasaweb]

I wondered the same. #3882 is already merged into main and set for Release 3.4 #2723 has been rejected from automatic merge and has to be reviewed manually before merging.

Both pull requests are merged into the master branch now. Does this mean that we can now hope?

[djpadz]

@dasaweb I just grabbed the latest nightly, and it looks like things are working! Huzzah!

[bretthl]

Same problem, ignore list not working.
Ubuntu 20.04
nautilus-nextcloud 3.3.6-20211028.151202.6d3270dd2-1.0~focal1

[dasaweb]

@bretthl Release 3.4.0 RC1 came out today (https://github.com/nextcloud/desktop/releases/tag/v3.4.0-rc1) that fixes #3882 and #2723. This should help. You can install this RC and help testing it ;-)

[bretthl]

@bretthl Release 3.4.0 RC1 came out today (https://github.com/nextcloud/desktop/releases/tag/v3.4.0-rc1) that fixes #3882 and #2723. This should help. You can install this RC and help testing it ;-)

Thanks for that, I will start using it and comment.

[bretthl]

3.4.1 RC1 is working with my testing using the ignore pattern somedir/* to suspend sync of all files in somedir. Keep up the good work, this app is awesome.

[LecrisUT]

Thank you for the pre-release. As expected, it works on MacOS as well with all the patterns in root and sub-directory being inherited correctly. Creating a .sync-exclude.lst overwrites the previous root one. Probably windows will work correctly as well, so any reason not to close the issue now that all changes have been merged?

Although, is there any #include ../.sync-exclude.lst functionality or similar?

[SMESH109]

using the ignore pattern somedir/* to suspend sync of all files in somedir.

isn't the correct pattern just somedir/ without the asterisk? won't somedir/* scan all the files in the subfolder anyway without syncing them afterwards instead of just leaving out the whole folder for scan?

[FlexW]

Although, is there any #include ../.sync-exclude.lst functionality or similar?

No

isn't the correct pattern just somedir/ without the asterisk? won't somedir/* scan all the files in the subfolder anyway without syncing them afterwards instead of just leaving out the whole folder for scan?

As far as my understand goes, somedir/ will ignore and don't sync the folder somedir. syncfolder/* sould still sync the folder somedir but no file inside it.

Closing this issue as it seems that the original issue is resolved. Thanks for your help!

[bretthl]

Although, is there any #include ../.sync-exclude.lst functionality or similar?

No

isn't the correct pattern just somedir/ without the asterisk? won't somedir/* scan all the files in the subfolder anyway without syncing them afterwards instead of just leaving out the whole folder for scan?

As far as my understand goes, somedir/ will ignore and don't sync the folder somedir. syncfolder/* sould still sync the folder somedir but no file inside it.

Closing this issue as it seems that the original issue is resolved. Thanks for your help!

Your are correct. I've been playing with both options and probably best to leave out the asterisk. Can anyone point me to a list of "flags" with explanations of each? Been awhile since I have used these.

[dasaweb]

@bretthl Release 3.4.0 RC1 came out today (https://github.com/nextcloud/desktop/releases/tag/v3.4.0-rc1) that fixes #3882 and #2723. This should help. You can install this RC and help testing it ;-)

I was now brave enough to test the 3.4.0 RC1 on Win11, and tataaaaaa: It works! Finally having an up-to-date Nextcloud client again after a year is great. THANK YOU to everyone who contributed here! 💖

[Detzi88]

I still have this issue on Version 3.4.2 (Ubuntu).

.git/*
.git/
.git

set in both the "general settings" and the "..." submenu are leading to the ".git" folder nor its content beeing ignored by nextcloud.

Is that only on my side or do others have the same issue?

[bop34]

Yeah me too

[Migsi]

I still have this issue on Version 3.4.2 (Ubuntu).

.git/*
.git/
.git

set in both the "general settings" and the "..." submenu are leading to the ".git" folder nor its content beeing ignored by nextcloud.

Is that only on my side or do others have the same issue?

Me as well, I'm on EndeavourOS, my client version is:

$ nextcloud --version
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
Nextcloud version 3.9.0git
Git revision c6c6c61f5e66e311abcd1e78ecec2266c2298dda
Using Qt 5.15.10, built against Qt 5.15.10
Using Qt platform plugin 'xcb'
Using 'OpenSSL 3.1.1 30 May 2023'
Running on EndeavourOS, x86_64

Steps to reproduce are:

1. Add "repo" and "repo/" to global and/or sync ignore list (mix as you wish, doesn't matter)
2. Move folder "repo" into synced directory
3. Give the client ~5 seconds
4. Watch it sync the excluded folder

I carefully read through all this thread and related resources to ensure I'm not just messing up my exclusion "pattern", but at this point I'm pretty sure I don't and this has to be broken again :(

[bop34]

I have some new info to share. I just got this working on Windows by using two ignores. For example to ignore everything in a folder named example I added both example/ and example\ to my ignore list. On Linux I was able to ignore with just example/. In both examples neither the folder itself nor the files in the folder synced to the server.
Notes:

• On Windows I got an error every sync if I did not do both example/ and example\, but the files still synced.
• Be aware that after ignoring a folder any previously synced stuff will remain on the server.
• I also noticed that after removing a folder from the ignore list any previously existing files would not sync and I had to recreate the whole folder and the files in it to get it to sync.

[mxwell-dev]

My observation is that .sync-exclude.lst files further down in the sync'd directory tree seems overwrite top level exclude rules, rather than extend on them. This may make sense if there's no way to negate a rule (in .gitignore with a prefixed !), but sure as rain is unintuitive. So if .git/ ignored top-level and some git repository synced by nextcloud contains a local exclude for build/, it will sync .git again.

[psyciknz]

Any further updates to this? I have a .sync-exclude.lst file in a subdirectory for excluding (as others are doing .git). If I have it in the global it exlcudes as I desire. But if the excludes are set in a .sync-exclude.lst where the .git directory is a subdirectory. Then I can see the sync log full of objects when I make a commit.

So for reference.
Nextcoud\Documents\Connections\.git is the .git folder I want to exclude everything but the config file.
Nextcoud\Documents\Connections\.sync-exclude.lst - is my exclude file (contents below)

].git/COMMIT_EDITMSG
].git/FETCH_HEAD
].git/HEAD
].git/ORIG_HEAD
.git/logs/
.git/logs/refs/
.git/logs/refs/heads/
.git/logs/refs/remotes/
.git/objects/
.git/objects/pack/
].git/objects/pack/tmp_pack*
.git/objects/*/
].git/objects/*/*.*
].git/objects/*/*
].git/objects/pack/*.*
].git/index.lock
.git/refs/
.git/refs/*/
.git/refs/*/*
.git/refs/tags/

Version 3.4.0 Windows.

[joshtrichards]

You're commenting on an Issue that has been closed out for several years. If experiencing something (even similar) please open a new dedicated Issue as it may have a different underlying cause.