Generative AI Evaluation v0.9 Release Notes

OPEA Release Notes v0.9

What’s New in OPEA v0.9

• Broaden functionality

  • Provide telemetry functionalities for metrics and tracing using Prometheus, Grafana, and Jaeger
  • Initialize two Agent examples: AgentQnA and DocIndexRetriever
  • Support for authentication and authorization
  • Add Nginx Component to strengthen backend security
  • Provide Toxicity Detection Microservice
  • Support the experimental Fine-tuning microservice

• Enhancement

  • Align the Microservice format with the standards of OpenAI (Chat Completions, Fine-tuning... etc)
  • Enhance the performance benchmarking and evaluation for GenAI Examples, ex: TGI, resource allocation, ...etc
  • Enable support for launching container images as a non-root user
  • Use Llama-Guard-2-8B as default Guardrails model and bge-large-zh-v1.5 as default embedding model, mistral-7b-grok as default CodeTrans model
  • Add ProductivitySuite to provide access management and maintains user context

• Deployment

  • Support Red Hat OpenShift Container Platform (RHOCP)
  • GenAI Microservices Connector (GMC) successfully tested on Nvidia GPUs
  • Add Kubernetes support for AudioQnA and VisualQnA examples

• OPEA Docker Hub: https://hub.docker.com/u/opea

• GitHub IO: https://opea-project.github.io/latest/index.html

• Thanks for the external contribution from Sharan Shirodkar, Aishwarya Ramasethu
  , Michal Nicpon and Jacob Mansdorfer

Details

GenAIExamples

• ChatQnA

  • Update port in set_env.sh(040d2b7)
  • Fix minor issue in ChatQnA Gaudi docker README(a5ed223)
  • update chatqna dataprep-redis port(02a1536)
  • Add support for .md file in file upload in the chatqna-ui(7a67298)
  • Added the ChatQnA delete feature, and updated the corresponding README(09a3196)
  • fixed ISSUE-528(45cf553)
  • Fix vLLM and vLLM-on-Ray UT bug(cfcac3f)
  • set OLLAMA_MODEL env to docker container(c297155)
  • Update guardrail docker file path(06c4484)
  • remove ray serve(c71bc68)
  • Refine docker_compose for dataprep param settings(3913c7b)
  • fix chatqna guardrails(db2d2bd)
  • Support ChatQnA pipeline without rerank microservice(a54ffd2)
  • Update the number of microservice replicas for OPEA v0.9(e6b4fff)
  • Update set_env.sh(9657f7b)
  • add env for chatqna vllm(f78aa9e)

• Deployment

  • update manifests for v0.9(ba78b4c)
  • Update K8S manifest for ChatQnA/CodeGen/CodeTrans/DocSum(01c1b75)
  • Update benchmark manifest to fix errors(4fd3517)
  • Update env for manifest(4fa37e7)
  • update manifests for v0.9(08f57fa)
  • Add AudioQnA example via GMC(c86cf85)
  • add k8s support for audioqna(0a6bad0)
  • Update mainifest for FaqGen(80e3e2a)
  • Add kubernetes support for VisualQnA(4f7fc39)
  • Add dataprep microservice to chatQnA example and the e2e test(1c23d87)

• Documentation

  • [doc] Update README.md(c73e4e0)
  • doc fix: Update README.md to remove specific dicscription of paragraph-1(5a9c109)
  • doc: fix markdown in docker_image_list.md(9277fe6)
  • doc: fix markdown in Translation/README.md(d645305)
  • doc: fix markdown in SearchQnA/README.md(c461b60)
  • doc: fix FaqGen/README.md markdown(704ec92)
  • doc: fix markdown in DocSum/README.md(83712b9)
  • doc: fix markdown in CodeTrans/README.md(076bca3)
  • doc: fix CodeGen/README.md markdown(33f8329)
  • doc: fix markdown in ChatQnA/README.md(015a2b1)
  • doc: fix headings in markdown files(21fab71)
  • doc: missed an H1 in the middle of a doc(4259240)
  • doc: remove use of HTML for table in README(e81e0e5)
  • Update ChatQnA readme with OpenShift instructions(ed48371)
  • Convert HTML to markdown format.(14621f8)
  • Fix typo {your_ip} to {host_ip}(ad8ca88)
  • README fix typo(abc02e1)
  • fix script issues in MD file(acdd712)
  • Minor documentation improvements in the CodeGen README(17b9676)
  • Refine Main README(08eb269)
  • [Doc]Add a micro/mega service WorkFlow for DocSum(343d614)
  • Update README for k8s deployment(fbb81b6)

• Other examples

  • Clean deprecated VisualQnA code(87617e7)
  • Using TGI official release docker image for intel cpu(b2771ad)
  • Add VisualQnA UI(923cf69)
  • fix container name(5ac77f7)
  • Add VisualQnA docker for both Gaudi and Xeon using TGI serving(2390920)
  • Remove LangSmith from Examples(88eeb0d)
  • Modify the language variable to match language highlight.(f08d411)
  • Remove deprecated folder.(7dd9952)
  • update env for manifest(4fa37e7)
  • AgentQnA example(67df280)
  • fix tgi xeon tag(6674832)
  • Add new DocIndexRetriever example(566cf93)
  • Add env params for chatqna xeon test(5d3950)
  • ProductivitySuite Combo Application with REACT UI and Keycloak Authen(947cbe3)
  • change codegen tgi model(06cb308)
  • change searchqna prompt(acbaaf8)
  • minor fix mismatched hf token(ac324a9)
  • fix translation gaudi env(4f3be23)
  • Minor fixes for CodeGen Xeon and Gaudi Kubernetes codegen.yaml (c25063f)

• CI/CD/UT

  • update deploy_gmc logical in cd workflow(c016d82)
  • fix ghcr.io/huggingface/text-generation-inference tag(503a1a9)
  • Add GMC e2e in CD workflow(f45e4c6)
  • Fix CI test changed file detect issue([5...

Generative AI Evaluation v0.8 Release Notes

OPEA Release Notes v0.8

What’s New in OPEA v0.8

• Broaden functionality

  • Support frequently asked questions (FAQs) generation GenAI example
  • Expand the support of LLMs such as Llama3.1 and Qwen2 and support LVMs such as llava
  • Enable end-to-end performance and accuracy benchmarking
  • Support the experimental Agent microservice
  • Support LLM serving on Ray

• Multi-platform support

  • Release the Docker images of GenAI components under OPEA dockerhub and support the deployment with Docker
  • Support cloud-native deployment through Kubernetes manifests and GenAI Microservices Connector (GMC)
  • Enable the experimental authentication and authorization support using JWT tokens
  • Validate ChatQnA on multiple platforms such as Xeon, Gaudi, AIPC, Nvidia, and AWS

• OPEA Docker Hub: https://hub.docker.com/u/opea

Details

GenAIExamples

• ChatQnA

  • Add ChatQnA instructions for AIPC(26d4ff)
  • Adapt Vllm response format (034541)
  • Update tgi version(5f52a1)
  • Update README.md(f9312b)
  • Udpate ChatQnA docker compose for Dataprep Update(335362)
  • [Doc] Add valid micro-service details(e878dc)
  • Updates for running ChatQnA + Conversational UI on Gaudi(89ddec)
  • Fix win PC issues(ba6541)
  • [Doc]Add ChatQnA Flow Chart(97da49)
  • Add guardrails in the ChatQnA pipeline(955159)
  • Fix a minor bug for chatqna in docker-compose(b46ae8)
  • Support vLLM/vLLM-on-Ray/Ray Serve for ChatQnA(631d84)
  • Added ChatQnA example using Qdrant retriever(c74564)
  • Update TEI version v1.5 for better performance(f4b4ac)
  • Update ChatQnA upload feature(598484)
  • Add auto truncate for embedding and rerank(8b6094)

• Deployment

  • Add Kubernetes manifest files for deploying DocSum(831463)
  • Update Kubernetes manifest files for CodeGen(2f9397)
  • Add Kubernetes manifest files for deploying CodeTrans(c9548d)
  • Updated READMEs for kubernetes example pipelines(c37d9c)
  • Update all examples yaml files of GMC in GenAIExample(290a74)
  • Doc: fix minor issue in GMC doc(d99461)
  • README for installing 4 worklods using helm chart(6e797f)
  • Update Kubernetes manifest files for deploying ChatQnA(665c46)
  • Add new example of SearchQnA for GenAIExample(21b7d1)
  • Add new example of Translation for GenAIExample(d0b028)

• Other examples

  • Update reranking microservice dockerfile path (d7a5b7)
  • Update tgi-gaudi version(3505bd)
  • Refine README of Examples(f73267)
  • Update READMEs(8ad7f3)
  • [CodeGen] Add codegen flowchart(377dd2)
  • Update audioqna image name(615f0d)
  • Add auto-truncate to gaudi tei (8d4209)
  • Update visualQnA chinese version(497895)
  • Fix Typo for Translation Example(95c13d)
  • FAQGen Megaservice(8c4a25)
  • Code-gen-react-ui(1b48e5)
  • Added doc sum react-ui(edf0d1)

• CI/UT

  • Frontend failed with unknown timeout issue (7ebe78)
  • Adding Chatqna Benchmark Test(11a56e)
  • Expand tgi connect timeout(ee0dcb)
  • Optimize gmc manifest e2e tests(15fc6f)
  • Add docker compose yaml print for test(bb4230)
  • Refactor translation ci test (b7975e)
  • Refactor searchqna ci test(ecf333)
  • Translate UT for UI(284d85)
  • Enhancement the codetrans e2e test(450efc)
  • Allow gmc e2e workflow to get secrets(f45f50)
  • Add checkout ref in gmc e2e workflow(62ae64)
  • SearchQnA UT(268d58)

GenAIComps

• Cores

  • Support https for microservice(2d6772)
  • Enlarge megaservice request timeout for supporting high concurrency(876ca5)
  • Add dynamic DAG(f2995a)

• LLM

  • Optional vllm microservice container build(963755)
  • Refine vllm instruction(6e2c28)
  • Introduce 'entrypoint.sh' for some Containers(9ecc5c)
  • Support llamaindex for retrieval microservice and remove langchain(61795f)
  • Update tgi with text-generation-inference:2.1.0(f23694)
  • Fix requirements(f4b029)
  • Add vLLM on Ray microservice(ec3b2e)
  • Update code/readme/UT for Ray Serve and VLLM([dd939c](https://gith...

Generative AI Evaluation v0.7 Release Notes

GenAIEvals

• Enable autorag to automatically generate the evaluation dataset and evaluate the RAG system(b24bff)
• Support document summarization evaluation with microservice(3ec544)
• Add RAGASMetric(7406bf)
• Update install bkc(26ddcc)

Others

• https://github.com/opea-project/GenAIExamples/releases/tag/v0.7

Generative AI Evaluation v0.6 Release Notes

GenAIEvals

• Add evaluating the models on text-generation tasks(lm-evaluation-harness)
• Add evaluating the models on coding tasks (bigcode-evaluation-harness)
• Add end-to-end evaluation with microservice

others

https://github.com/opea-project/GenAIExamples/releases/tag/v0.6