Llama 3 Inference with Warp

[guoriyue]

Hi Warp community, I built a llama 3 inference from scratch using Warp. Check out the repo here – feedbacks are welcome!

[mmacklin]

This is amazing! Thank you @guoriyue !

[guoriyue]

Late update: I tried tiling but saw little performance gain, likely because our LLM kernels are already SIMT. Also torch.compile slightly worsened performance, I'm still unsure about the reason. I just used torch.compile but I have no idea how it actually works