[BUG]: Indexing does not finish (desktop application)

[Woisek]

I tried the desktop variant of this repo and when it comes to indexing, it "works" but it stays at "Confirm rebuilding image index?" and never finishes.

Any suggestions? 🤔

[zanllp]

Could you please check if the scanning path includes any excessively large folders, such as the entire C drive? This could be a potential cause for the issue.

[Woisek]

Could you please check if the scanning path includes any excessively large folders, such as the entire C drive? This could be a potential cause for the issue.

Thanks for getting back.
I don't use the C drive (would be a bad habit).
With "excessively large folders", what do you mean by that? Regarding path length or content in them? 🤔

Edit: I checked the amount of the two folders I use. One has 8800 images, the other, 14920.

[zanllp]

Can you try updating to version 0.7.0 and see if this issue is resolved?

[Woisek]

Can you try updating to version 0.7.0 and see if this issue is resolved?

Thanks for the update. Unfortunately, it's still says "An error occurred". Rebuild image index -> Start
At least it would be helpful if we would know what error occurred.

Plus, now my ESET Internet Security believes that iib_api_server.exe is a variant of Python/PSW.Agent_AGen.R Trojaner

[zanllp]

I will add some loggers in the future to make debugging more convenient

Plus, now my ESET Internet Security believes that iib_api_server.exe is a variant of Python/PSW.Agent_AGen.R Trojaner

dot't worry. see #47 (comment)

[Woisek]

I will add some loggers in the future to make debugging more convenient

This would be helpful, thanks.

Plus, now my ESET Internet Security believes that iib_api_server.exe is a variant of Python/PSW.Agent_AGen.R Trojaner

dot't worry. see #47 (comment)

Yeah, I just wanted to point this out, I didn't believe it anyway. 😅

[Woisek]

@zanllp
Anything new about this issue? I noticed, that I really would need the search function with my ever-growing amount of generated images. 😐

[zanllp]

You can try backing up the iib.db file first, then delete the iib.db file to allow IIB to generate a new index file. If any unexpected issues occur, you can restore the backed-up file.

After deleting the iib.db file, make sure to restart the software for the changes to take effect.

[Woisek]

I did as you suggested, but unfortunately it hasn't changed anything.

May I suggest, to alter the code in that way, so that we get an actual usable error message? Right now, this is absolutely insufficient.
And a question: Under Settings -> Client-specific settings -> Initiate software startup configuration -> Read Stable Diffusion Webui Config
What folder exactly is here required?
And this -> If you have installed sd-webui and this extension, it is recommended to use this option to directly read the configuration and share data.
Does that mean, I have to install the IIB extension for A1111 to use this standalone program? 🤔

[zanllp]

Firstly, IIB is capable of running independently. Reading the sd-webui folder is for the purpose of data sharing; if you don't have sd-webui, you can simply ignore this step. Then, you don't necessarily need to 'rebuild index'; it's meant to remove some tags that haven't been indexed, which is usually not required. If you can search normally, there's no need to use this feature. Indeed, the error messages are not sufficient, and we plan to enhance them in future updates.

[zanllp]

Remove iib.db -> Restart IIB -> Go to the image page and wait for the image index to be generated (do not go directly to the global settings page to rebuild the index).

[Woisek]

if you don't have sd-webui, you can simply ignore this step.

Actually, I have A1111 and also SD:Next. Still I have no clue, what folder could be meant.

If you can search normally, there's no need to use this feature.

That is now the question for me: Isn't indexing the base for searching? Indexing, in the sense of gathering all the found images' informations to be able to search efficiently? 🤔

Anyway, I tried your suggestion and now I have 50% success.
As I mentioned already, I use 2 folders. Adding the "smaller" one and fuzzy search it, works now.
But the other one throws again an error (An error occurred).

This folder is ~19GB in size and has 8799 files in 577 folders (inclusive subfolders).
Edit: The root and subfolder begins with an underscore, could this be problematic?

Any suggestions on that, please? 🤔

[zanllp]

Generating an index is a prerequisite for searching, but when you activate the search function, IIB will determine whether to update the index based on the modification dates of the folders. It then performs an incremental update on the index, and this process doesn't require your intervention. I suggest you temporarily forget about the "rebuild index" functionality.

[Woisek]

I suggest you temporarily forget about the "rebuild index" functionality.

Alright, I can do that.
How about the problem with the other folder, basically my "end storage" of images? How can that be solved?

[zanllp]

Actually, I have A1111 and also SD:Next. Still I have no clue, what folder could be meant.

Please refer to the image below and select the folder from the a1111-sd-webui Git repository that you are using.

20231229_003132.mp4

[zanllp]

How about you give it a try using the method mentioned above first? In a while, I will compile a new version with log.

[Woisek]

Can't play the video, it's marked as 'private' ... ?
Anyway, you mean selecting the root of sd-webui. But this returns also an error:

But for this to work, it probably needs the webui extension installed, right?

In a while, I will compile a new version with log.

Thanks, that's great. Besides this all, I have some suggestions that maybe would be useful to add/implement. I guess you want this as a feature request ... ?

[zanllp]

https://github.com/zanllp/sd-webui-infinite-image-browsing/releases/download/untagged-c964d6fccdc0e8110090/Infinite.Image.Browsing_0.7.1_x64-setup.exe

log file

[Woisek]

Thanks, but I get a 404 not found when clicking the link ... 🤔

[zanllp]

Infinite.Image.Browsing_0.7.1_x64-setup.zip

[Woisek]

Thanks, that worked now. Her is the content of the log after I installed and run it trying to search the folder:

2023-12-29 10:34:52,410 - scripts.iib.logger - INFO - Received request: GET http://127.0.0.1:60970/infinite_image_browsing/global_setting
2023-12-29 10:34:52,547 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:60970/infinite_image_browsing/check_path_exists
2023-12-29 10:35:04,933 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:60970/infinite_image_browsing/db/get_image_tags
2023-12-29 10:35:10,030 - scripts.iib.logger - INFO - Received request: GET http://127.0.0.1:60970/infinite_image_browsing/db/basic_info
2023-12-29 10:35:10,509 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data
2023-12-29 10:35:10,630 - scripts.iib.logger - ERROR - An exception occurred while processing POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data: 'charmap' codec can't encode character '\ufeff' in position 111: character maps to
2023-12-29 10:35:10,631 - scripts.iib.logger - ERROR - An exception occurred while processing POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data: 'NoneType' object is not callable
2023-12-29 10:35:13,933 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data
2023-12-29 10:35:13,965 - scripts.iib.logger - ERROR - An exception occurred while processing POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data: 'charmap' codec can't encode character '\ufeff' in position 111: character maps to
2023-12-29 10:35:13,965 - scripts.iib.logger - ERROR - An exception occurred while processing POST http://127.0.0.1:60970/infinite_image_browsing/db/update_image_data: 'NoneType' object is not callable
2023-12-29 10:35:35,877 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:60970/infinite_image_browsing/shutdown
2023-12-29 10:35:37,334 - scripts.iib.logger - INFO - Received request: GET http://127.0.0.1:61062/infinite_image_browsing/global_setting
2023-12-29 10:35:37,879 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:61062/infinite_image_browsing/check_path_exists
2023-12-29 10:35:58,676 - scripts.iib.logger - INFO - Received request: POST http://127.0.0.1:61062/infinite_image_browsing/shutdown

[zanllp]

Infinite.Image.Browsing_0.7.2_x64-setup.zip

[Woisek]

Many thanks for this update, it seems to work now.
I now opened a feature request here -> #487 for some qol things.
Thanks! 👍