Skip to content

Commit

Permalink
Add IPEX documentation (huggingface#828)
Browse files Browse the repository at this point in the history
* change readme, source/index, source/installation

* add ipex doc 1st step

* update readme for command line usage

* fix bug for ipex readme

* add export doc

* update all ipex docs

* rm diffusers

* change register

* Update README.md

Co-authored-by: Ella Charlaix <[email protected]>

* Update docs/source/installation.mdx

Co-authored-by: Ella Charlaix <[email protected]>

* fix readme

* fix ipex exporter args comments

* extend ipex export explain

* fix ipex reference.mdx

* add comments for auto doc

* rm cli export

* Update optimum/commands/export/ipex.py

Co-authored-by: Ella Charlaix <[email protected]>

* rm commit hash in export command

* rm export

* rm jit

* add ipex on doc's docker file

* indicate that ipex model only supports for cpu and the export format will be changed to compile in the future

* Update docs/source/ipex/inference.mdx

Co-authored-by: Ella Charlaix <[email protected]>

* explain patching

* rm ipex reference

* Update docs/source/ipex/inference.mdx

* Update docs/source/ipex/inference.mdx

* Update docs/source/ipex/inference.mdx

* Update docs/source/index.mdx

* Update docs/source/ipex/inference.mdx

* Update docs/source/ipex/models.mdx

* Update docs/Dockerfile

---------

Co-authored-by: Ella Charlaix <[email protected]>
  • Loading branch information
jiqing-feng and echarlaix authored Aug 27, 2024
1 parent 1f3d0c2 commit 403c696
Show file tree
Hide file tree
Showing 9 changed files with 162 additions and 4 deletions.
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,15 +223,14 @@ To load your IPEX model, you can just replace your `AutoModelForXxx` class with
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("He's a dreadful magician and")

```

For more details, please refer to the [documentation](https://intel.github.io/intel-extension-for-pytorch/#introduction).


## Running the examples

Check out the [`examples`](https://github.com/huggingface/optimum-intel/tree/main/examples) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.
Check out the [`examples`](https://github.com/huggingface/optimum-intel/tree/main/examples) and [`notebooks`](https://github.com/huggingface/optimum-intel/tree/main/notebooks) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.

Do not forget to install requirements for every example:

Expand Down
11 changes: 11 additions & 0 deletions docs/source/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,16 @@
title: Tutorials
isExpanded: false
title: OpenVINO
- sections:
- local: ipex/inference
title: Inference
- local: ipex/models
title: Supported Models
- sections:
- local: ipex/tutorials/notebooks
title: Notebooks
title: Tutorials
isExpanded: false
title: IPEX
title: Optimum Intel
isExpanded: false
2 changes: 2 additions & 0 deletions docs/source/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ limitations under the License.

🤗 Optimum Intel is the interface between the 🤗 Transformers and Diffusers libraries and the different tools and libraries provided by Intel to accelerate end-to-end pipelines on Intel architectures.

[Intel Extension for PyTorch](https://intel.github.io/intel-extension-for-pytorch/#introduction) (IPEX) is an open-source library which provides optimizations for both eager mode and graph mode, however, compared to eager mode, graph mode in PyTorch* normally yields better performance from optimization techniques, such as operation fusion.

[Intel Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html) is an open-source library enabling the usage of the most popular compression techniques such as quantization, pruning and knowledge distillation. It supports automatic accuracy-driven tuning strategies in order for users to easily generate quantized model. The users can easily apply static, dynamic and aware-training quantization approaches while giving an expected accuracy criteria. It also supports different weight pruning techniques enabling the creation of pruned model giving a predefined sparsity target.

[OpenVINO](https://docs.openvino.ai) is an open-source toolkit that enables high performance inference capabilities for Intel CPUs, GPUs, and special DL inference accelerators ([see](https://docs.openvino.ai/2024/about-openvino/compatibility-and-support/supported-devices.html) the full list of supported devices). It is supplied with a set of tools to optimize your models with compression techniques such as quantization, pruning and knowledge distillation. Optimum Intel provides a simple interface to optimize your Transformers and Diffusers models, convert them to the OpenVINO Intermediate Representation (IR) format and run inference using OpenVINO Runtime.
Expand Down
3 changes: 2 additions & 1 deletion docs/source/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ To install the latest release of 🤗 Optimum Intel with the corresponding requi
|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------|
| [Intel Neural Compressor (INC)](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html) | `pip install --upgrade --upgrade-strategy eager "optimum[neural-compressor]"`|
| [Intel OpenVINO](https://docs.openvino.ai ) | `pip install --upgrade --upgrade-strategy eager "optimum[openvino]"` |
| [Intel Extension for PyTorch](https://intel.github.io/intel-extension-for-pytorch/#introduction) | `pip install --upgrade --upgrade-strategy eager "optimum[ipex]"` |

The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version.

Expand All @@ -42,4 +43,4 @@ or to install from source including dependencies:
python -m pip install "optimum-intel[extras]"@git+https://github.com/huggingface/optimum-intel.git
```

where `extras` can be one or more of `neural-compressor`, `openvino`, `nncf`.
where `extras` can be one or more of `neural-compressor`, `openvino`, `ipex`.
45 changes: 45 additions & 0 deletions docs/source/ipex/inference.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Inference

Optimum Intel can be used to load models from the [Hub](https://huggingface.co/models) and create pipelines to run inference with IPEX optimizations (including patching with custom operators, weight prepacking and graph mode) on a variety of Intel processors. For now support is only enabled for CPUs.


## Loading

You can load your model and apply IPEX optimizations (including weight prepacking and graph mode). For supported architectures like LLaMA, BERT and ViT, further optimizations will be applied by patching the model to use custom operators.
For now, support is only enabled for CPUs and the original model will be exported via TorchScript. In the future `torch.compile` will be used and model exported via TorchScript will get deprecated.

```diff
import torch
from transformers import AutoTokenizer, pipeline
- from transformers import AutoModelForCausalLM
+ from optimum.intel import IPEXModelForCausalLM

model_id = "gpt2"
- model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
+ model = IPEXModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, export=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("He's a dreadful magician and")
```

As shown in the table below, each task is associated with a class enabling to automatically load your model.

| Auto Class | Task |
|--------------------------------------|--------------------------------------|
| `IPEXModelForSequenceClassification` | `text-classification` |
| `IPEXModelForTokenClassification` | `token-classification` |
| `IPEXModelForQuestionAnswering` | `question-answering` |
| `IPEXModelForImageClassification` | `image-classification` |
| `IPEXModel` | `feature-extraction` |
| `IPEXModelForMaskedLM` | `fill-mask` |
| `IPEXModelForAudioClassification` | `audio-classification` |
| `IPEXModelForCausalLM` | `text-generation` |
46 changes: 46 additions & 0 deletions docs/source/ipex/models.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Supported models

🤗 Optimum provides IPEX optimizations for both eager mode and graph mode. It provides classes and functions to perform this step easily.
Here is the list of the supported architectures :

## [Transformers](https://huggingface.co/docs/transformers/index)

- Albert
- Bart
- Beit
- Bert
- BlenderBot
- BlenderBotSmall
- Bloom
- CodeGen
- DistilBert
- Electra
- Flaubert
- GPT-2
- GPT-BigCode
- GPT-Neo
- GPT-NeoX
- Llama
- MPT
- Mistral
- MobileNet v1
- MobileNet v2
- MobileVit
- OPT
- ResNet
- Roberta
- Roformer
- SqueezeBert
- UniSpeech
- Vit
- Wav2Vec2
- XLM
16 changes: 16 additions & 0 deletions docs/source/ipex/tutorials/notebooks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Notebooks

## Inference

| Notebook | Description | | |
|:---------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|------:|
| [How to run inference with the IPEX](https://github.com/huggingface/optimum-intel/tree/main/notebooks/ipex) | Explains how to export your model to IPEX and to run inference with IPEX model on text-generation task | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/optimum-intel/blob/main/notebooks/ipex/text_generation.ipynb) | [![Open in AWS Studio](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/huggingface/optimum-intel/blob/main/notebooks/ipex/text_generation.ipynb) |
36 changes: 35 additions & 1 deletion optimum/intel/ipex/modeling_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,14 +198,48 @@ def _from_pretrained(
token: Optional[Union[bool, str]] = None,
revision: Optional[str] = None,
force_download: bool = False,
cache_dir: str = HUGGINGFACE_HUB_CACHE,
cache_dir: Union[str, Path] = HUGGINGFACE_HUB_CACHE,
subfolder: str = "",
local_files_only: bool = False,
torch_dtype: Optional[Union[str, "torch.dtype"]] = None,
trust_remote_code: bool = False,
file_name: Optional[str] = WEIGHTS_NAME,
**kwargs,
):
"""
Loads a model and its configuration file from a directory or the HF Hub.
Arguments:
model_id (`str` or `Path`):
The directory from which to load the model.
Can be either:
- The model id of a pretrained model hosted inside a model repo on huggingface.co.
- The path to a directory containing the model weights.
use_auth_token (Optional[Union[bool, str]], defaults to `None`):
Deprecated. Please use `token` instead.
token (Optional[Union[bool, str]], defaults to `None`):
The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
when running `huggingface-cli login` (stored in `~/.huggingface`).
revision (`str`, *optional*):
The specific model version to use. It can be a branch name, a tag name, or a commit id.
force_download (`bool`, defaults to `False`):
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.
cache_dir (`Union[str, Path]`, *optional*):
The path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.
subfolder (`str`, *optional*)
In case the relevant files are located inside a subfolder of the model repo either locally or on huggingface.co, you can specify the folder name here.
local_files_only (`bool`, *optional*, defaults to `False`):
Whether or not to only look at local files (i.e., do not try to download the model).
torch_dtype (`Optional[Union[str, "torch.dtype"]]`, *optional*)
float16 or bfloat16 or float32: load in a specified dtype, ignoring the model config.torch_dtype if one exists. If not specified, the model will get loaded in float32.
trust_remote_code (`bool`, *optional*)
Allows to use custom code for the modeling hosted in the model repository. This option should only be set for repositories you trust and in which you have read the code, as it will execute on your local machine arbitrary code present in the model repository.
file_name (`str`, *optional*):
The file name of the model to load. Overwrites the default file name and allows one to load the model
with a different name.
"""
if use_auth_token is not None:
warnings.warn(
"The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.",
Expand Down
4 changes: 4 additions & 0 deletions optimum/intel/ipex/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,12 @@


_HEAD_TO_AUTOMODELS = {
"feature-extraction": "IPEXModel",
"text-generation": "IPEXModelForCausalLM",
"text-classification": "IPEXModelForSequenceClassification",
"token-classification": "IPEXModelForTokenClassification",
"question-answering": "IPEXModelForQuestionAnswering",
"fill-mask": "IPEXModelForMaskedLM",
"image-classification": "IPEXModelForImageClassification",
"audio-classification": "IPEXModelForAudioClassification",
}

0 comments on commit 403c696

Please sign in to comment.