THIS PROJECT IS ARCHIVED
Intel will not provide or guarantee development of or support for this project, including but not limited to, maintenance, bug fixes, new releases or updates. Patches to this project are no longer accepted by Intel.
If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the community, please create your own fork of the project.

LiveAvatar

Live conversion of monocular video streams to 3D Gaussian head avatars

Live cross-identity animation from a single image

Table of Contents

• Installation
• Quick Start
• Streaming server
• Training

Installation

Minimum tested requirements (older versions might still work):

• Python 3.12
• Cuda 12.6
• PyTorch 2.7
• Ubuntu 24.04

Clone the project

git clone https://github.com/IntelLabs/LiveAvatar.git
cd LiveAvatar

Install Python packages

pip install -r requirements.txt

Install INRIA diff-gaussian-renderer

pip install git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git --no-build-isolation

See here for details: https://github.com/graphdeco-inria/diff-gaussian-rasterization

Install PyTorch3D

Follow the installation guide on https://pytorch3d.org.

git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d && pip install .

If you encounter problems during install, try:

pip install . --no-build-isolation

Download LiveAvatar model weights

• We are preparing the release of the model file

Quick Start

Run the demo

After installation, test with the provided demo:

# Basic demo with default images
python demo.py

# Specify custom images
python demo.py --target_image="path/to/target.jpg" --exp_image="path/to/expression.jpg"

# Use provided demo faces
python demo.py --target_image="assets/demo_faces/mona_lisa.jpg" --exp_image="assets/demo_faces/gothic_woman.png"

Expected Output:

• Window 1: Target face (identity source)
• Window 2: Expression source face
• Window 3: Animated 3D avatar (rotating point cloud)

Running the Streaming Server (Webcam)

For real-time avatar generation with webcam:

# Use webcam input with GUI (default)
python streaming_server.py

# Use video file as input
python streaming_server.py --input video --video_file="path/to/video.mp4"

# Disable GUI (headless mode)
python streaming_server.py --no-gui

Expected Output:

• Window 1: [Tracker] Input frame - Your webcam feed with face detection
• Window 2: LiveAvatar - Real-time 3D avatar rendering

Interactive Controls:

• Space - Cycle through reference images (blend your face with Mona Lisa, Vincent van Gogh, etc.)
• C - Clear reference (use only your own face)
• 0-9 - Select different camera viewpoints
• Arrow Keys - Next/prev. reference image
• Mouse drag - Rotate camera
• +/- - Adjust camera distance
• Q - Quit application

Additional Options:

# Stream over network socket
python streaming_server.py --input socket --host 127.0.0.1 --port_recv 9000

# Custom camera settings
python streaming_server.py --camera-azimuth 45.0 --camera-elevation 15.0

# Use specific reference image
python streaming_server.py --ref-img-id 0  # 0 = Mona Lisa, 1 = Vincent

Training

Install pre-processing libraries

Face segmentation: face-parsing.Pytorch

git clone https://github.com/zllrunning/face-parsing.PyTorch.git libs/face_parsing_pytorch

Download the face-farsing.PyTorch pre-trained model and save it in ./libs/face_parsing_pytorch/res/cp

Background removal: RobustVideoMatting

git clone https://github.com/PeterL1n/RobustVideoMatting.git libs/RobustVideoMatting

Download the RobustVideoMatting pre-trained model and save it in ./libs/RobustVideoMatting

Training data setup

CelebV-HQ

Run face tracker to perform frame extraction, video matting, facial keypoint detection, and camera pose estimation on input videos in 'CelebV-HQ/35666':

python datasets/parallel_prepare_celebvhq.py --st 0 --None --show False --root ./data/datasets/CelebV-HQ --output_dir ./data/datasets/CelebV-HQ/processed_celebvhq

The directory ./CelebV-HQ should have the following structure

CelebV-HQ/
- celebvhq_info.json
- 35666/
- processed_celebvhq/
    - frames/
    - alpha/
    - seg/
    - poses/

Option 1

Use config parameter 'celebvhq_root' to specify dataset location.

train.py --celebvhq_root=/mydata/CelebV-HQ

Option 2

Place training data in ./data/datasets or create symlink to pointing to training data location:

ln -s </foo/bar/data> ./data

with CelebV-HQ located in /foo/bar/data/datasets/CelebV-HQ.