Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matrix Multiplication Interface Replacement Advisory #2268

Open
jackwenshann opened this issue Jun 28, 2024 · 0 comments
Open

Matrix Multiplication Interface Replacement Advisory #2268

jackwenshann opened this issue Jun 28, 2024 · 0 comments

Comments

@jackwenshann
Copy link

Hello, please advise. I want to replace the matrix calculation in the ggml_compute_forward_mul_mat whisper.cpp part with the https://github.com/marty1885/llama.cpp/blob/rknpu2-backend/ggml-rknpu2.c matrix multiplication, I modified pack->ordered_data to src0->data, Finally I found out that the reasoning was wrong, and I tried to use void matmul(const float* A, const float* B, float* C, int M, int K, int N) {
for (int i = 0; I < M; ++i) {
for (int j = 0; j < N; ++j) {
C[i * N + j] = 0;
for (int k = 0; k < K; ++k) {
C[i * N + j] += A[i * K + k] * B[k * N + j];
}
}
}
Replace the calculation part of the CPU in the ggml_compute_forward_mul_mat, and the inference result is also incorrect. How can I directly do inference calculations in a matmul-like way, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant