关于Expert数量限制的一些问题 #2749
Unanswered
sitabulaixizawaluduo
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
`float max_logit = 0.f;
// Find top-k
PRAGMA_UNROLL
for (int k = 0; k < top_k; ++k) {
int max_bit = 0;
float max_val = -std::numeric_limits::infinity();
int bit = 1;
for (int e = 0; e < experts; ++e) {
const auto val = logits[ti * experts + e];
// const auto val = logits[e * tokens + ti];
if ((mask & bit) && val > max_val) {
max_bit = bit;
max_val = val;
}
bit *= 2;
}
mask -= max_bit;
if (k == 0) {
max_logit = max_val;
}
}`
从这段代码看,是不是意味着能够支持最大的expert数量只有32?感觉解答
Beta Was this translation helpful? Give feedback.
All reactions