Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Fix] Fix the chunked prefill condition (#2628)
This PR fixes a bug of the prefill chunking which may cause the running batch size exceeding the maximum allowed batch size.
- Loading branch information