Skip to content

Commit

Permalink
Merge pull request #73 from OpenNetLab/dev
Browse files Browse the repository at this point in the history
  • Loading branch information
Pterosaur authored Mar 30, 2021
2 parents f5422b2 + 5aae8be commit f0de4f2
Show file tree
Hide file tree
Showing 16 changed files with 413 additions and 24 deletions.
12 changes: 10 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ output_dir := out/Default
target_dir := target
target_lib_dir := $(target_dir)/lib
target_bin_dir := $(target_dir)/bin
target_pylib_dir := $(target_dir)/pylib

compile_docker := alphartc-compile
release_docker := alphartc
Expand Down Expand Up @@ -37,7 +38,8 @@ peerconnection_serverless:
make docker-$@ \
output_dir=$(output_dir) \
target_lib_dir=$(target_lib_dir) \
target_bin_dir=$(target_bin_dir)
target_bin_dir=$(target_bin_dir) \
target_pylib_dir=$(target_pylib_dir)

# Docker internal command

Expand All @@ -51,8 +53,14 @@ docker-app: docker-peerconnection_serverless

docker-peerconnection_serverless:
ninja -C $(output_dir) peerconnection_serverless

mkdir -p $(target_lib_dir)
cp modules/third_party/onnxinfer/lib/*.so $(target_lib_dir)
cp modules/third_party/onnxinfer/lib/*.so.* $(target_lib_dir)

mkdir -p $(target_bin_dir)
cp $(output_dir)/peerconnection_serverless $(target_bin_dir)
cp $(output_dir)/peerconnection_serverless $(target_bin_dir)/peerconnection_serverless.origin
cp examples/peerconnection/serverless/peerconnection_serverless $(target_bin_dir)

mkdir -p $(target_pylib_dir)
cp modules/third_party/cmdinfer/*.py $(target_pylib_dir)/
57 changes: 51 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,12 @@

AlphaRTC is a fork of Google's WebRTC project using ML-based bandwidth estimation, delivered by the OpenNetLab team. By equipping WebRTC with a more accurate bandwidth estimator, our mission is to eventually increase the quality of transmission.

AlphaRTC replaces Google Congestion Control (GCC) with ONNXInfer, an ML-powered bandwidth estimator, which takes in an ONNX model to make bandwidth estimation more accurate. ONNXInfer is proudly powered by Microsoft's [ONNXRuntime](https://github.com/microsoft/onnxruntime).
AlphaRTC replaces Google Congestion Control (GCC) with two customized congestion control interfaces, PyInfer and ONNXInfer. The PyInfer provides an opportunity to load external bandwidth estimator written by Python. The external bandwidth estimator could be based on ML framework, like PyTorch or TensorFlow, or a pure Python algorithm without any dependencies. And the ONNXInfer is an ML-powered bandwidth estimator, which takes in an ONNX model to make bandwidth estimation more accurate. ONNXInfer is proudly powered by Microsoft's [ONNXRuntime](https://github.com/microsoft/onnxruntime).

## Environment

**We recommend you directly fetch the pre-provided Docker images from [Github release](https://github.com/OpenNetLab/AlphaRTC/releases/latest/download/alphartc.tar.gz)**

Ubuntu 18.04 is the only officially supported distro at this moment. For other distros, you may be able to compile your own binary, or use our pre-provided Docker images.

## Compilation
Expand Down Expand Up @@ -107,7 +109,7 @@ Note: all commands below work for both Linux (sh) and Windows (pwsh), unless oth
gn gen out/Default
```
5. Comile
5. Compile
```shell
ninja -C out/Default peerconnection_serverless
```
Expand Down Expand Up @@ -151,9 +153,6 @@ This section describes required fields for the json configuration file.
- **bwe_feedback_duration**: The duration the receiver sends its estimated target rate every time(*in millisecond*)
- **onnx**
- **onnx_model_path**: The path of the [onnx](https://www.onnxruntime.ai/) model
- **video_source**
- **video_disabled**:
- **enabled**: If set to `true`, the client will not take any video source as input
Expand Down Expand Up @@ -188,11 +187,57 @@ This section describes required fields for the json configuration file.
- **fps**: Frames per second of the output video file
- **file_path**: The file path of the output video file in YUV format
#### Use PyInfer or ONNXInfer
##### PyInfer
The default bandwidth estimator is PyInfer, You should implement your Python class named `Estimator` with required methods `report_states` and `get_estimated_bandwidth` in Python file `BandwidthEstimator.py ` and put this file in your workspace.
There is an example of Estimator with fixed estimated bandwidth 1Mbps. Here is an example [BandwidthEstimator.py](examples/peerconnection/serverless/corpus/BandwidthEstimator.py).
```python
class Estimator(object):
def report_states(self, stats: dict):
'''
stats is a dict with the following items
{
"send_time_ms": uint,
"arrival_time_ms": uint,
"payload_type": int,
"sequence_number": uint,
"ssrc": int,
"padding_length": uint,
"header_length": uint,
"payload_size": uint
}
'''
pass
def get_estimated_bandwidth(self)->int:
return int(1e6) # 1Mbps
```
##### ONNXInfer
If you want to use the ONNXInfer as the bandwidth estimator, you should specify the path of onnx model in the config file. Here is an example configuration [receiver.json](examples/peerconnection/serverless/corpus/receiver.json)
- **onnx**
- **onnx_model_path**: The path of the [onnx](https://www.onnxruntime.ai/) model
#### Run peerconnection_serverless
- Dockerized environment
To better demonstrate the usage of peerconnection_serverless, we provide an all-inclusive corpus in `examples/peerconnection/serverless/corpus`. You can use the following commands to execute a tiny example. After these commands terminates, you will get `outvideo.yuv` and `outaudio.wav`.
PyInfer:
```shell
sudo docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc alphartc peerconnection_serverless receiver_pyinfer.json
sudo docker exec alphartc peerconnection_serverless sender_pyinfer.json
```
ONNXInfer:
``` shell
sudo docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc alphartc peerconnection_serverless receiver.json
sudo docker exec alphartc peerconnection_serverless sender.json
Expand Down
8 changes: 4 additions & 4 deletions api/alphacc_config.cc
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,10 @@ bool ParseAlphaCCConfig(const std::string& file_path) {
RETURN_ON_FAIL(
GetInt(top, "bwe_feedback_duration", &config->bwe_feedback_duration_ms));

RETURN_ON_FAIL(GetValue(top, "onnx", &second));
RETURN_ON_FAIL(
GetString(second, "onnx_model_path", &config->onnx_model_path));
second.clear();
if (GetValue(top, "onnx", &second)) {
GetString(second, "onnx_model_path", &config->onnx_model_path);
second.clear();
}

bool enabled = false;
RETURN_ON_FAIL(GetValue(top, "video_source", &second));
Expand Down
25 changes: 25 additions & 0 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,28 @@ steps:
- script: docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc alphartc peerconnection_serverless receiver.json
&& docker exec alphartc peerconnection_serverless sender.json
displayName: 'run example'

- script: docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc_pyinfer alphartc peerconnection_serverless receiver_pyinfer.json
&& docker exec alphartc_pyinfer peerconnection_serverless sender_pyinfer.json
displayName: 'run pyinfer example'

- script: docker save alphartc | gzip > alphartc.tar.gz
displayName: "Export alphartc docker image"

- publish: $(System.DefaultWorkingDirectory)/alphartc.tar.gz
continueOnError: true
artifact: alphartc.tar.gz
displayName: "Archive AlphaRTC Peerconnection"

- task: GitHubRelease@0
inputs:
gitHubConnection: 'Pterosaur (1)'
repositoryName: '$(Build.Repository.Name)'
action: 'create'
tagSource: manual
tag: $(Build.BuildNumber)
title: alphartc
assets: '$(System.DefaultWorkingDirectory)/alphartc.tar.gz'
changeLogCompareToRelease: 'lastFullRelease'
changeLogType: 'commitBased'
displayName: "Release target"
3 changes: 2 additions & 1 deletion dockers/Dockerfile.release
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
FROM ubuntu:18.04

RUN apt-get update && apt-get install -y \
libx11-6 libgomp1
libx11-6 libgomp1 python3

COPY lib /usr/lib/

COPY bin /usr/bin/

COPY pylib /usr/lib/python3/dist-packages/
20 changes: 20 additions & 0 deletions examples/peerconnection/serverless/corpus/BandwidthEstimator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@

class Estimator(object):
def report_states(self, stats: dict):
'''
stats is a dict with the following items
{
"send_time_ms": uint,
"arrival_time_ms": uint,
"payload_type": int,
"sequence_number": uint,
"ssrc": int,
"padding_length": uint,
"header_length": uint,
"payload_size": uint
}
'''
pass

def get_estimated_bandwidth(self)->int:
return int(1e6) # 1Mbps
54 changes: 54 additions & 0 deletions examples/peerconnection/serverless/corpus/receiver_pyinfer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"serverless_connection": {
"autoclose": 20,
"sender": {
"enabled": false
},
"receiver": {
"enabled": true,
"listening_ip": "0.0.0.0",
"listening_port": 8000
}
},
"bwe_feedback_duration": 200,
"video_source": {
"video_disabled": {
"enabled": true
},
"webcam": {
"enabled": false
},
"video_file": {
"enabled": false,
"height": 240,
"width": 320,
"fps": 10,
"file_path": "testmedia/test.yuv"
}
},
"audio_source": {
"microphone": {
"enabled": false
},
"audio_file": {
"enabled": true,
"file_path": "testmedia/test.wav"
}
},
"save_to_file": {
"enabled": true,
"audio": {
"file_path": "outaudio.wav"
},
"video": {
"width": 320,
"height": 240,
"fps": 10,
"file_path": "outvideo.yuv"
}
},
"logging": {
"enabled": true,
"log_output_path": "webrtc.log"
}
}
44 changes: 44 additions & 0 deletions examples/peerconnection/serverless/corpus/sender_pyinfer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"serverless_connection": {
"autoclose": 20,
"sender": {
"enabled": true,
"dest_ip": "0.0.0.0",
"dest_port": 8000
},
"receiver": {
"enabled": false
}
},
"bwe_feedback_duration": 200,
"video_source": {
"video_disabled": {
"enabled": false
},
"webcam": {
"enabled": false
},
"video_file": {
"enabled": true,
"height": 240,
"width": 320,
"fps": 10,
"file_path": "testmedia/test.yuv"
}
},
"audio_source": {
"microphone": {
"enabled": false
},
"audio_file": {
"enabled": true,
"file_path": "testmedia/test.wav"
}
},
"save_to_file": {
"enabled": false
},
"logging": {
"enabled": false
}
}
44 changes: 44 additions & 0 deletions examples/peerconnection/serverless/peerconnection_serverless
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys
import os
import subprocess
import traceback
import json

sys.path.append(os.getcwd())

import cmdinfer


def main():
app = subprocess.Popen(
["peerconnection_serverless.origin"] + sys.argv[1:],
bufsize=1,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
try:
cmdinfer.main(app.stdout, app.stdin)
app.wait()
except:
app.terminate()
app.wait()
error_message = traceback.format_exc()
error_message = "\n{}".format(error_message)
sys.stderr.write(error_message)
if len(sys.argv[1:]) == 0:
return
config_file = sys.argv[1]
config_file = json.load(open(config_file, "r"))
if "logging" not in config_file:
return
if "enabled" not in config_file["logging"] or not config_file["logging"]["enabled"]:
return
with open(config_file["logging"]["log_output_path"], "a") as log_file:
log_file.write(error_message)


if __name__ == "__main__":
main()
3 changes: 2 additions & 1 deletion modules/remote_bitrate_estimator/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ rtc_library("remote_bitrate_estimator") {
"../../system_wrappers:metrics",
"//third_party/abseil-cpp/absl/strings",
"//third_party/abseil-cpp/absl/types:optional",
"//modules/third_party/statcollect:stat_collect"
"//modules/third_party/statcollect:stat_collect",
"//modules/third_party/cmdinfer:cmdinfer"
]

if (is_linux) {
Expand Down
Loading

0 comments on commit f0de4f2

Please sign in to comment.