[Feature] do sample = False?

[boqiny]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

I am testing the sglang with qwen2, but the output seems to be unstable even if I have set sampling params;

temp = 0,
top_k = 1

But I got inconsistent outputs, while naive HF implementation always gives me same output when set do_sample = False,
is there such feature in SGL? Is this due to some non-deterministic Cuda operations? Thanks

Related resources

No response