list of open-source publicly-available llms for code

[andre15silva]

	Name	Publication Date	Model Type	Sizes	URL
-	CodeGen	03/22	Decoder	350M, 2B, 6B, 16B	https://huggingface.co/Salesforce/codegen-16B-mono
-	InCoder	04/22	Decoder	1.3B, 6.7B	https://huggingface.co/facebook/incoder-6B
-	CodeGeeX	09/22	Decoder	13B	https://huggingface.co/spaces/THUDM/CodeGeeX
-	santacoder				https://huggingface.co/bigcode/santacoder
-	replit				https://huggingface.co/replit/replit-code-v1_5-3b
-	codet5				https://huggingface.co/Salesforce/codet5-large
-	plbart				https://huggingface.co/models?other=plbart

[monperrus]

diff-codegen-350m
diff-codegen-2b
diff-codegen-6b

all fine-tuned from Salesforce’s CodeGen code synthesis models

ref: https://carper.ai/diff-models-a-new-way-to-edit-code/

[monperrus]

GPT-J / GPT-J-6B

demo at https://6b.eleuther.ai/
https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b

used by https://arxiv.org/pdf/2208.08289.pdf

[andre15silva]

to merge: https://github.com/eugeneyan/open-llms (section "Open LLMs for code")

[andre15silva]

codegen2 (also supports infilling)

https://github.com/salesforce/CodeGen2

[monperrus]

https://github.com/bigcode-project/starcoder
15.5B parameter model supports code generation and infilling

[monperrus]

StableCode by StableAI
https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b

[monperrus]

WizardCoder
https://huggingface.co/WizardLM/WizardCoder-15B-V1.0

[monperrus]

code-llama by Meta
https://about.fb.com/news/2023/08/code-llama-ai-for-coding/

Code Llama: Open Foundation Models for Code
https://arxiv.org/pdf/2308.12950

[monperrus]

The Mistral models https://mistral.ai/

@martinezmatias says they are good.

Mistral 7B
https://arxiv.org/pdf/2310.06825

[monperrus]

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
https://arxiv.org/pdf/2310.06266

[monperrus]

Qwen

CODE-QWEN and CODE-QWEN-CHAT 通义千问 (Alibaba)

QWEN TECHNICAL REPORT
https://arxiv.org/pdf/2309.16609.pdf
https://github.com/QwenLM/Qwen

Qwen2. 5-Coder Technical Report
https://arxiv.org/pdf/2409.12186

Nov 2024:

Qwen 2.5-Coder-32B-Instruct Performance: @Alibaba_Qwen announced Qwen 2.5-Coder-32B-Instruct, which matches or surpasses GPT-4o on multiple coding benchmarks. Early testers reported it as "indistinguishable from o1-preview results" (@hrishioa) and noted its competitive performance in code generation and reasoning.

updated @andre15silva

[monperrus]

For the record: CodeTrans

• paper CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing
• model https://github.com/agemagician/CodeTrans

[monperrus]

DeepSeek Coder: Let the Code Write Itself

• https://deepseekcoder.github.io/
• https://github.com/deepseek-ai/DeepSeek-Coder

[monperrus]

Magicoder
Magicoder: Source Code Is All You Need
https://arxiv.org/abs/2312.02120
https://huggingface.co/TheBloke/Magicoder-S-DS-6.7B-GGUF

[monperrus]

StarCoder2
https://drive.google.com/file/d/17iGn3c-sYNiLyRSY-A85QOzgzGnGiVI3/view

[monperrus]

CodeShell Technical Report
https://arxiv.org/pdf/2403.15747

CodeShell-Base, a seven billion-parameter foundation model with 8K context length, showcasing exceptional proficiency in code comprehension, which outperforms CodeLlama in Humaneval after training on just 500 billion tokens (5 epochs).

[monperrus]

Mixtral, @FredBonux is able to use it over groq

[andre15silva]

mistralai/Codestral-22B-v0.1

https://huggingface.co/mistralai/Codestral-22B-v0.1

[monperrus]

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
https://arxiv.org/pdf/2406.11931

[monperrus]

aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Completion
https://www.semanticscholar.org/reader/2c5dd0f56eff1caa3edb20354374a9585181ea73

[monperrus]

Tencent 's Hunyuan (huggingface, paper)

[monperrus]

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
https://arxiv.org/pdf/2411.04905