[BUG] Dual GPU displaying incorrect VRAM

[eeljpg]

Environment

🪟 Windows

System

Windows 11 23H2

Describe the problem

I have both an iGPU and a dedicated eGPU. The launcher displays my VRAM as the iGPU vram while still recognizing the eGPU. It says:
"GPU: NVIDIA GeForce RTX 3090 - VRAM: 4 GB".
Problem does not exist in automatic1111 sd or task manager.
For full disclosure, the GPU is an egpu through Oculink. Still works fine and all 24gb vram are usable. Not looking to use iGPU vram. LLMs still utilize 24gb of 3090 VRAM, so it seems to be a display error only.

Additional info

SillyTavern: 1.12.4
STL: 24.1.3.0
GPU: NVIDIA GeForce RTX 3090 - VRAM: 4 GB

Please tick the boxes

• I have explained the issue clearly, and I included all relevant info
• I have checked that this issue hasn't already been raised
• I have checked the docs

[BlueprintCoding]

@eeljpg pushed a fix for this. Not 100% sure it will work as my desktop does not have integrated graphics but I'm pretty confident it will work. Let me know if works or not.

c7825ee

[eeljpg]

Updated, still displaying GPU: NVIDIA GeForce RTX 3090 - VRAM: 4 GB
I noticed the change involved adding an Intel graphics check, my iGPU is AMD. It comes up in task manager as "AMD Radeon 780M Graphics"

[BlueprintCoding]

@eeljpg Please let me know if the newest update works.
8bb3875

[BlueprintCoding]

Issue will be closed in a few days if no response.

[eeljpg]

Hello, my bad. Issue is still present on latest.
Looking at the changes, it seems the check added was for Intel(R) HD Graphics, but I have an AMD iGPU.

[BlueprintCoding]

Unfortunately the core of this issue relates to external GPUs and without an eGPU to be able to test on I'm not sure I will be able to create a fix for this. There is no way I can find to detect AMD iGPUs in the registry or wmic without finding a list of every AMD iGPU and hardcoding a check against each of them.

Since this is not an issue that will effect 99% of users and doesn't prevent the launcher from working, just doesn't show correct VRAM which is a "quality of life" feature. I will be closing this issue.

[JaceEthaniel]

I also had this problem.

For reference, I have an AMD processor with a built in GPU, and I also have an NVIDIA RTX 4070TI Super...
The original script was only ever selecting the first iterated GPU, so this changes it so it will iterate through all the GPUs.

One of the errors was that the "AdapterRAM > 0" would always return true, because most iGPUs have more than 0 VRAM...

For example: my return of the wmic command shows me to have 536870912 AdapterRAM on my AMD Radeon(TM) Graphics entry; which is greater than 0, and so it was being chosen.

To fix that, I changed it to this and it works for me now (this could very much be done better, but I made these changes after searching on google for an hour on how to write in batch... so excuse the mess):

FILENAME: gpu_info.bat

@echo off

REM Initialize variables
setlocal enabledelayedexpansion
set /a iteration=0
set /a last_VRAM=0
set "last_GPU=Unknown"
set "GPU_name=Unknown"
set "VRAM=Unknown"
set "primary_gpu_found=false"
REM Detect GPU and store name, excluding integrated GPUs if discrete GPUs are found
for /f "tokens=2 delims==" %%f in ('wmic path Win32_VideoController get name /value ^| find "="') do (
	REM If GPU name is blank, set it; else, if it is not blank and does not equal the current value of f,
	REM move gpu name to last_gpu and set gpu_name to the new value. This is here so that if the current card has less
	REM VRAM than the previous card, we can set the name back properly. I doubt it would be needed... but you never know.
	if /i !GPU_name! equ "" (
		set "GPU_name=%%f"
	) else if !GPU_name! neq %%f (
		set "last_GPU=!GPU_name!"
		set "GPU_name=%%f"
	)
	REM Run PowerShell command to retrieve VRAM size and divide by 1GB
	REM This was moved here because if we leave the first for loop it wont continue iterating.
	REM Instead, it will break the loop and end the script. This way it keeps going.
	for /f "usebackq tokens=*" %%i in (`powershell -Command "$qwMemorySize = (Get-ItemProperty -Path 'HKLM:\SYSTEM\ControlSet001\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0*' -Name HardwareInformation.qwMemorySize -ErrorAction SilentlyContinue).'HardwareInformation.qwMemorySize'; if ($null -ne $qwMemorySize -and $qwMemorySize -is [array]) { $qwMemorySize = [double]$qwMemorySize[!iteration!] } else { $qwMemorySize = [double]$qwMemorySize }; if ($null -ne $qwMemorySize) { [math]::Round($qwMemorySize/1GB) } else { 'Property not found' }"`) do (
		set "VRAM=%%i"
	)
	REM iterate so we can move through the objects in the powershell script...
	set /a iteration=!iteration!+1
	REM If the VRAM is greater than 0 (it always should be, but just in case...) AND the lastVRAM is not the same as VRAM
	REM (might have to change this in the case where two cards have the same VRAM, but the user wants the first card...
	REM in which case we would want to see if it is the first iteration or not... but I'm not here to do all your work!)
	REM AND the lastVRAM is Grt than VRAM... set it to the previous card. We don't have to check the other way, because
	REM we already set VRAM. This way takes more builds into account... but not all.
	if /i !lastVRAM! gtr 0 (
		if /i !lastVRAM! neq !VRAM! (
			if /i !lastVRAM! gtr !VRAM! (
				set "VRAM=!lastVRAM!"
				set "GPU_name=!last_GPU!"
			)
		)
	) else (
		set "lastVRAM=!VRAM!"
	)
)

REM Display GPU name and VRAM
echo GPU: %cyan_fg_strong%%GPU_name%%reset% - VRAM: %cyan_fg_strong%%VRAM%%reset% GB
endlocal

With that being said, when I tried to then download a model for TabbyAPI, it said I didn't have enough VRAM, so there are more places that need to be fixed than just the gpu_info.bat file.

[JaceEthaniel]

It is the launcher.bat that needs fixing too... It uses the same non-iterating powershell script to get the VRAM for the GPU, which only returns the first GPU (which is always an iGPU if it exists.)

So I replaced the part that normally finds the VRAM (at lines 431-433) with this and it works now:

set /a iteration=0
set /a last_UVRAM=0
REM Detect GPU and store name, excluding integrated GPUs if discrete GPUs are found
for /f "tokens=2 delims==" %%f in ('wmic path Win32_VideoController get name /value ^| find "="') do (
	REM Run PowerShell command to retrieve VRAM size and divide by 1GB
	for /f "usebackq tokens=*" %%i in (`powershell -Command "$qwMemorySize = (Get-ItemProperty -Path 'HKLM:\SYSTEM\ControlSet001\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0*' -Name HardwareInformation.qwMemorySize -ErrorAction SilentlyContinue).'HardwareInformation.qwMemorySize'; if ($null -ne $qwMemorySize -and $qwMemorySize -is [array]) { $qwMemorySize = [double]$qwMemorySize[!iteration!] } else { $qwMemorySize = [double]$qwMemorySize }; if ($null -ne $qwMemorySize) { [math]::Round($qwMemorySize/1GB) } else { 'Property not found' }"`) do (
		set "UVRAM=%%i"
	)
	set /a iteration=!iteration!+1
	REM If the VRAM is greater than 0 (it always should be, but just in case...) AND the lastUVRAM is not the same as VRAM
	REM (might have to change this in the case where two cards have the same VRAM, but the user wants the first card...
	REM in which case we would want to see if it is the first iteration or not... but I'm not here to do all your work!)
	REM AND the lastUVRAM is Grt than VRAM... set it to the previous card. We don't have to check the other way, because
	REM we already set VRAM. This way takes more builds into account... but not all.
	if /i !lastUVRAM! gtr 0 (
		if /i !lastUVRAM! neq !UVRAM! (
			if /i !lastUVRAM! gtr !UVRAM! (
				set "UVRAM=!lastUVRAM!"
				set "GPU_name=!last_GPU!"
			)
		)
	) else (
		set "lastUVRAM=!UVRAM!"
	)
)

[deffcolony]

@JaceEthaniel If you believe this change improves the launcher for everyone, I recommend you to make a pull request. This way, your contribution can be properly reviewed and integrated, and your name will automatically be added to the contributor list. It's a better approach than posting the fixed code in a comment, it provides proper credit for your work.

[JaceEthaniel]

@eeljpg does the problem still exist for you?

[eeljpg]

@eeljpg does the problem still exist for you?

So, funny news. My GPU died so I don't have an eGPU setup anymore and I built a whole new PC. Still have a different non-external 3090 though. As my hardware is entirely different now and there isn't an egpu, I wasn't expecting the issue to still exist, but it seems I carry a special level of edge-case energy as it now displays "GPU: Parsec Virtual Display Adapter - VRAM: 24 GB". Weirdly, it persists in showing this even if I disable the parsec adapter in device manager.

[JaceEthaniel]

I see, so you still have the driver installed and it is being detected as an actual GPU... Unfortunately I am not well-versed enough to solve that particular problem with an auto-select method like what is currently in place. I would suggest trying to uninstall that driver if you don't expect to use it any time soon.
I have been quite busy with my classes, but I plan on eventually altering it so that you can choose the GPU you are using (and then, of course, making a pull request); that wont be for a while though...

Basically, my plan is to have it list all the GPUs you have on the first start up (it will check the GPU log file to see if it exists, and if not it will ask which GPU you will be using), then in the toolbox I am going to add an option to select a different GPU. Ultimately though it is primarily a visual thing. The only thing it actually affects is the launcher itself deciding if you have enough VRAM to run certain models. The actual image/text generators will use the correct GPU/driver (well, unless you are REALLY good at getting those edge-case issues).