Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.

Official code for "Heterogeneous Graph-based Framework with Disentangled Representations Learning for Multi-target Cross Domain Recommendation"

Official code for "Learning Positional Attention for Sequential Recommendation"

Official code for "Controllable Talking Face Generation by Implicit Facial Keypoints Editing"

【grps服务样例】包含c++、python自定义工程以及快捷部署样例。

【grps接入vllm】通过vllm LLMEngine Api实现LLM服务。

网易新闻推荐模型推理优化相关

easy-ngo是由网易传媒开发的基于Go语言的开发工具包，基于easy-ngo工具包，开发者可以快速构建高可用、大并发的应用。