[feat] Add math eval to CI

[XiaotongJiang]

Issue #2504

• Compared SGL and vLLM's math eval score to address concerns on SGL being constant low on this eval (res linked below)
• Add math eval to CI

Setup:

• Prompt: simple eval from openai
• Grading Model: gemini-1.5-flash
• Temperature 0.0

inference engine	Model	# Examples	score	model official score
vllm	llama 3.2 1b	500	0.248	0.306
sgl	llama 3.2 1b	500	0.248	0.306
vllm	llama 3.2 3b	500	0.476	0.48
sgl	llama 3.2 3b	500	0.484	0.48
vllm	llama 3.1 8b	500	0.50	0.519
sgl	llama 3.1 8b	500	0.496	0.519
sgl	llama 3.1 8b	65	0.492	0.519

[zhyncs]

Thanks!!

[zhaochenyang20]

@XiaotongJiang @zhyncs @merrymercy @shuaills I am talking with XIaotong and will take a look on this. Also, thanks so much. this having been suspended for so long 😂

[XiaotongJiang]

sorry for the disruption

AssertionError: 0.505 not greater than or equal to 0.509

need to loose this assertion a bit