Skip to content

Rebase on top of b6298 #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 146 commits into
base: main-upstream
Choose a base branch
from
Open

Rebase on top of b6298 #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
5094941
src: reduce the logging
kpouget Apr 9, 2025
23409dc
Add helper scripts
kpouget Apr 9, 2025
1183af9
build-system: integrate the Remoting Frontend backend build
kpouget Apr 9, 2025
7c8b8fb
ggml: ggml-remotingfrontend: stubs of a new backend
kpouget Apr 9, 2025
f39b99f
.github: remove
kpouget Apr 9, 2025
226c539
CMakeLists: add the ggml files and include Mesa files
kpouget Apr 10, 2025
5336ac9
ggml-*: move the ggml interfaces to a dedicated file
kpouget Apr 10, 2025
88d94c3
run.vulkan.sh: allow running with GDB
kpouget Apr 10, 2025
b0ce82d
virtgpu: start integrating virt-gpu code
kpouget Apr 10, 2025
7ccd1b1
virtgpu: allocate a shared page with the host
kpouget Apr 10, 2025
a864dab
run.remoting: cleanup the screen before running
kpouget Apr 11, 2025
1288908
Reduce the verbose logging
kpouget Apr 11, 2025
2d20b2d
Trace the executionpath
kpouget May 2, 2025
f002dcc
virtgpu: abort early
kpouget May 2, 2025
b497545
virtgpu: add the virtgpu_submit to kick a command on the host
kpouget May 2, 2025
a2ada88
podman_compile.sh: add compile helper
kpouget May 2, 2025
354e55d
virtgpu: move the logging functions to virtgpu-utils
kpouget May 5, 2025
44e62f0
virtgpu: use venus CS functions
kpouget May 5, 2025
cd9f3e9
virtgpu: make more generic
kpouget May 6, 2025
3c98263
ggml-remotingfrontend: fix and make more generic
kpouget May 6, 2025
d286d69
prepare.backend.sh: helper script
kpouget May 6, 2025
ad0c43a
build.backend.sh: helper script
kpouget May 6, 2025
2c63dbd
build: integrate the remoting-backend skeleton
kpouget May 6, 2025
8679435
remoting: start using shared header files
kpouget May 6, 2025
9c66722
remotingbackend/CMakeLists: add header dependencies
kpouget May 6, 2025
790d84b
ggml-remotingbackend: add skeleton of argument passing
kpouget May 6, 2025
646cddd
remotingfrontend: improve the typing
kpouget May 6, 2025
f6b8a40
podman_compile: delete the pod before compiling
kpouget May 6, 2025
134ff4b
virtgpu-utils: add WARNING
kpouget May 7, 2025
326a80d
virtgpu: split the remote call into prepare/call/finish
kpouget May 7, 2025
0e2ae43
ggml-backend-reg: reindent
kpouget May 7, 2025
567e2a8
move thks_bye() to virtgpu-utils
kpouget May 7, 2025
a0acf8d
virtgpu: remove forward call wip code
kpouget May 7, 2025
54327f3
ggml-remotingfrontend: build the apir framework
kpouget May 7, 2025
29527a2
ggml-remotingbackend: build the apir framework
kpouget May 7, 2025
48b2328
Add support for device name and description
kpouget May 13, 2025
faf7789
ggml: src: ggml-metal/ggml-metal: make less verbose
kpouget May 13, 2025
584401d
ggml-remotingbackend: include the ggml backend initialization
kpouget May 13, 2025
6d10bf0
remoting: include device_get_type and device_get_memory
kpouget May 13, 2025
cec6854
ggml: src: ggml-remotingbackend/backend: make less verbose
kpouget May 13, 2025
4830b28
shared: venus_cs: add more CS functions
kpouget May 13, 2025
fbb510a
ggml: src: ggml-remotingfrontend/ggml-remoting: make the NOT_IMPLEMEN…
kpouget May 13, 2025
081d7b9
ggml: src: ggml-remotingfrontend/virtgpu-forward: make less verbose
kpouget May 13, 2025
5683479
remoting: correct the device_get_* name order
kpouget May 13, 2025
473a70c
remoting: add support for device_supports_op
kpouget May 13, 2025
a966f26
ggml/src/ggml-remotingbackend/shared/venus_cs.h: clearer message when…
kpouget May 13, 2025
35ad53c
ggml/src/ggml-remotingfrontend/virtgpu.cpp: make less verbose
kpouget May 13, 2025
63c5c52
remoting: reindent and mark functions as NOT_IMPLEMENTED
kpouget May 13, 2025
c6db63e
Add buffer-type support
kpouget May 14, 2025
17009ba
Keep working
kpouget May 14, 2025
ab6524f
Keep working on buffer types and buffers
kpouget May 15, 2025
8f06b44
implemnt alloc_buffer and get_base
kpouget May 15, 2025
6e2adf6
buffer: clean ups
kpouget May 15, 2025
0ffa4bf
Keep working on buffers
kpouget May 15, 2025
1e484a1
build.backend: build llama-run
kpouget May 16, 2025
dbe47f8
ggml: src: ggml-remotingbackend/shared/venus_cs: fix memory corruptio…
kpouget May 16, 2025
2e49088
ggml: src: ggml-remotingfrontend/ggml-backend-device: handcode the caps
kpouget May 16, 2025
a088b6c
remoting: implement buffer_set_tensor
kpouget May 16, 2025
72e15e7
remoting: improve
kpouget May 16, 2025
959179f
remotingbackend: accept the virgl context argument
kpouget May 19, 2025
87491f8
remotingfrontend: implement buffer_set_tensor with a guest shared page
kpouget May 19, 2025
c638bd1
ggml: src: ggml-remotingbackend/backend-dispatched-buffer: implement …
kpouget May 19, 2025
8d0ef1c
remotingfrontend: add more STOP_HERE calls
kpouget May 19, 2025
cddfa5d
remotingfrontend: add IMPLEMENTED_ONCE
kpouget May 19, 2025
4761586
ggml: src: ggml-remotingfrontend/virtgpu-shm: reduce the verbosity
kpouget May 19, 2025
70bed26
ggml: src: ggml-remotingfrontend/ggml-backend-reg: refactor to untigh…
kpouget May 19, 2025
3312f5d
ggml: src: ggml-remotingfrontend/ggml-remoting: remove draft code
kpouget May 19, 2025
5bcb9dc
remotingfrontend: add host buffer memory allocation
kpouget May 19, 2025
c9da129
remoting: add clear buffer and get_tensor
kpouget May 20, 2025
e95189e
remoting: add skeleton for graph_compute method
kpouget May 20, 2025
f3fab8c
remoting: continue the compute_graph skeleton
kpouget May 20, 2025
7f4e9cb
Continue the skeleton
kpouget May 20, 2025
586c3a9
remoting: recursively encode/decode the tensors
kpouget May 20, 2025
5f910a8
keep working
kpouget May 20, 2025
5fb1607
start using the ggml-rpc serialization methods
kpouget May 21, 2025
4e54111
remoting: implement the free_buffer function
kpouget May 21, 2025
4dff7f0
remoting: highlight the hot path
kpouget May 21, 2025
14daf76
remoting: fix the warnings and mute the debug logs when not in debug …
kpouget May 22, 2025
5d9641e
scripts: make it easier to build and run in prod mode
kpouget May 22, 2025
18da4cf
remotingfrontend: always prepare a shared memory for data
kpouget May 22, 2025
91f60df
remoting: release device buffers on exit
kpouget May 22, 2025
d41e7f3
remoting: refactor the buffer context
kpouget May 26, 2025
7026e8c
remoting: exchange more data
kpouget May 26, 2025
1bef318
podman_compile: pass the PERF_MODE flag to the container
kpouget May 26, 2025
7734728
examples: run: run: measure the generation throughput
kpouget May 27, 2025
5d80465
examples: run: run: stop after 25 tokens
kpouget May 27, 2025
32641f4
remoting: add basic timing measurements
kpouget May 27, 2025
50d345f
remoting: cleanup the logs
kpouget May 27, 2025
38cc90f
ggml: src: ggml-remotingfrontend/ggml-backend-reg: call the initializ…
kpouget May 27, 2025
f4ac17d
disable APIR_ALLOC_FROM_HOST_PTR
kpouget May 27, 2025
01c5fc1
remoting: cache the buffer_get_base result
kpouget May 27, 2025
e3975e6
examples: run: run: improve the timing measurement
kpouget May 28, 2025
b801838
examples: run: run: remove the stop after 25 tokens
kpouget May 28, 2025
eefb58e
remoting: improve the timing measurement
kpouget May 28, 2025
027ef7a
remoting: allow compiling to Vulkan
kpouget May 28, 2025
0b80159
ggml: src: ggml-remotingfrontend/virtgpu: reduce the response time wa…
kpouget May 28, 2025
654f52f
remoting: experiement with buffer_from_ptr
kpouget Jun 2, 2025
4ddf33b
remoting: remove from_ptr code
kpouget Jun 2, 2025
5a4a38a
remoting: try host_pointer
kpouget Jun 2, 2025
892e570
remoting: try from_host_ptr
kpouget Jun 2, 2025
a022d81
remoting: make alloc_memory + alloc_from_host_ptr work :)
kpouget Jun 2, 2025
714db8e
build.backend: export SDKROOT to please apple compiler ...
kpouget Jun 11, 2025
d67aed7
prepare.backend.sh: more flags
kpouget Jun 11, 2025
f540ac3
run.vulkan.sh: more flexible
kpouget Jun 11, 2025
d38911f
run.remoting.sh: more flexible
kpouget Jun 11, 2025
b63a2d7
prepare.vulkan.sh: more details
kpouget Jun 11, 2025
9f7525f
ggml: src: ggml-remotingfrontend/virtgpu: don't include virglrenderer…
kpouget Jun 11, 2025
a32d8a4
ggml: src: ggml-remotingfrontend/virtgpu: don't use absolute paths in…
kpouget Jun 11, 2025
f44cdf9
remoting: rewrite to avoid hard-coded paths
kpouget Jun 11, 2025
496a80b
update the custom scripts
kpouget Jun 11, 2025
31352ed
ggml: src: ggml-remotingfrontend/virtgpu-shm: import the cpp atomic
kpouget Jun 17, 2025
8fd37a5
remoting: reintroduce the support for support_op(tensor)
kpouget Jun 19, 2025
91ad061
remotingbackend: add an optional call to support_op to avoid crashing…
kpouget Jun 19, 2025
dde12b1
remotingfrontend: reduce and cleanup the logging
kpouget Jun 19, 2025
0eb86c3
remotingfrontend: cache some values
kpouget Jun 19, 2025
aa55111
Update the custom scripts
kpouget Jun 19, 2025
43766d9
remotingbackend: set APIR_DEVICE_SUPPORTS_OP_ALWAYS_TRUE = 1
kpouget Jun 19, 2025
a23f4d7
remotingfrontend: use GGML_ASSERT instead of assert
kpouget Jun 19, 2025
6aeff90
tools: run: run: use GGML_ASSERT instead of assert
kpouget Jun 19, 2025
1df6f69
ggml-metal: expose some internal device properties
kpouget Jun 25, 2025
01d5c51
remoting: add support for running the ggml-metal support_op method in…
kpouget Jun 25, 2025
32bfb94
ggml: src: ggml-remotingfrontend/virtgpu-forward-buffer: remove an un…
kpouget Jun 25, 2025
d0383ef
remotingfrontend: detect initialization issue
kpouget Jun 27, 2025
eae3907
remotingfrontend: detect initialization issue
kpouget Jun 27, 2025
62c97d3
ggml: src: ggml-remotingbackend/shared/apir_backend: add apir_backend…
kpouget Jul 1, 2025
9c03318
ggml: src: ggml-remotingfrontend/virtgpu: give more info on error
kpouget Jul 1, 2025
2292e2e
ggml: src: ggml-remotingbackend/backend-dispatched-metal: add missing…
kpouget Jul 1, 2025
28280f1
ggml-remotingbackend: allow saving the hypervisor logs to a file
kpouget Jul 1, 2025
516933f
remotingbackend: update the VIRGL_APIR indexes to match virlrenderer …
kpouget Jul 2, 2025
87d71f2
remotingfrontend/virtgpu: give more time to load the libraries and lo…
kpouget Jul 4, 2025
3bd97bc
OWNERS: add file for openshift CI
kpouget Jul 4, 2025
1fa1c2d
ggml-remotingbackend/shared/apir_backend: return the duration in stop…
kpouget Jul 7, 2025
d1b255f
ggml-remotingfrontend/virtgpu: rewrite the timeout mechanism to make …
kpouget Jul 7, 2025
38d49bd
ggml-remotingfrontend: improve the timers display
kpouget Jul 7, 2025
2a2b19b
ggml-remotingfrontend: add an ERROR log level
kpouget Jul 7, 2025
2ec8784
ggml-remotingfrontend: turn some INFO logs into MESSAGE (always printed)
kpouget Jul 7, 2025
2f8f3ee
ggml-remotingfrontend: turn a INFO log into ERROR
kpouget Jul 7, 2025
135ff21
ggml: src: ggml-remotingfrontend/virtgpu: correctly fail when the vir…
kpouget Jul 7, 2025
ebb5fb2
run.remoting: update to run llama-server
kpouget Jul 7, 2025
cb0fca5
OWNERS: Update
kpouget Jul 9, 2025
910e3fc
remoting: improve the frontend<>backend error handling
kpouget Jul 8, 2025
3b9b455
remoting: improve the frontend<>backend return code exchange
kpouget Jul 9, 2025
8725fdd
ggml: src: ggml-remotingfrontend/virtgpu: fix typo
kpouget Aug 20, 2025
871c4c8
update the build scripts
kpouget Aug 27, 2025
e5c6771
ggml: src: ggml-remotingfrontend/ggml-metal-remoting: update ggml_met…
kpouget Aug 29, 2025
2e4899d
update the build scripts
kpouget Aug 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 0 additions & 87 deletions .github/ISSUE_TEMPLATE/010-bug-compilation.yml
View file
Open in desktop

This file was deleted.

101 changes: 0 additions & 101 deletions .github/ISSUE_TEMPLATE/011-bug-results.yml
View file
Open in desktop

This file was deleted.

91 changes: 0 additions & 91 deletions .github/ISSUE_TEMPLATE/019-bug-misc.yml
View file
Open in desktop

This file was deleted.

51 changes: 0 additions & 51 deletions .github/ISSUE_TEMPLATE/020-enhancement.yml
View file
Open in desktop

This file was deleted.

52 changes: 0 additions & 52 deletions .github/ISSUE_TEMPLATE/030-research.yml
View file
Open in desktop

This file was deleted.

Loading