Highlights
Improvements
Examples
Bug Fixing
Highlights
- Support vLLM CPU and IPEX CPU WOQ with Transformer-like API
- Support Streamingllm on Habana Gaudi
Improvements
- Add true_sequential for WOQ GPTQ (091f564 )
- Refine GPU scripts to support OOB mode (f4b3a7b )
- QBits adapt to the latest BesTLA (c169bec )
- Improve CPU WOQ scheme setting (fd3ee5cf1 )
- Enhance voicechat API with multilang tts streaming support (98daf37d )
- Add bias internal convertion in qbits (7c29f6f1 )
- Add DynamicQuantConfig, QuantAwareTrainingConfig and StaticQuantConfig (6a15b48, e1f4666d)
Examples
- Integrate EAGLE with ITREX (e559929d )
Bug Fixing
- Fix for token latency (ae7a4ae )
- Fix phi3 quantization scripts (2af19c7a6 )
- Fix is_intel_gpu_available (47d5024 )
- Fix QLoRA CPU issue due to internal API change (699ffca )
- Add scale and weight dtype check for quantization config (307c1a8b )
- Fix tf autodistill bug of transformers>=4.37 (8116fbb2 )
Validated Configurations
- Python 3.10
- Ubuntu 22.04
- PyTorch 2.2.0+cpu
- Intel® Extension for Torch 2.2.0+cpu