Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

大家都是用多大内存运行web_demo.py的啊 #87

Open
anshiwangwang opened this issue Apr 14, 2024 · 4 comments
Open

大家都是用多大内存运行web_demo.py的啊 #87

anshiwangwang opened this issue Apr 14, 2024 · 4 comments

Comments

@anshiwangwang
Copy link

anshiwangwang commented Apr 14, 2024 •
edited
Loading

服务器32G的内存，每次运行都是imagebind能加载，一到Vicuna就被killed，看了后台应该是内存爆了。想问下有什么办法能解决一下嘛？万分感谢！！！
Uploading 屏幕截图 2024-04-14 160833.png…
@We-IOT
Copy link

We-IOT commented May 14, 2024

同问
Initializing language decoder from ../pretrained_ckpt/vicuna_ckpt/7b_v0 ...
Killed

@We-IOT
Copy link

We-IOT commented May 14, 2024

看了源码，每次都是使用cpu加载，最后把多个模型加载完，再到cuda上，确实比较消耗内存，我的也是32G，确实在/vicuna_ckpt/7b_v0加载中就爆内存了。
解决方法也简单，增大swapfile就可以了。
目前在在完成后会到cuda，按照demo中模型加载后占用gpu接近20个G。

@ggamaz
Copy link

ggamaz commented May 20, 2024

显存使用很少，但内存使用很大

@jovejnn
Copy link

jovejnn commented Jul 8, 2024

有具体增加swapfile的教程吗，我增加了但是还是加载vicuna时被kill了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ggamaz @jovejnn @anshiwangwang @We-IOT