This is an API which is using xnect to calculate dynamic pose data. Input data is a rgb video. It can process multiple people.
The backend of our application utilizes a range of technologies and libraries to ensure efficient processing and data handling.
- Flask: A flexible WSGI web application framework.
- Flask-HTTPAuth: Simplifies HTTP authentication in Flask.
- Flask-HTTPBasicAuth: Basic HTTP authentication for Flask.
- Flask-SQLAlchemy: Adds SQLAlchemy support to Flask.
- Flask-restplus: Facilitates building REST APIs with Flask.
- gunicorn: A robust Python WSGI HTTP Server for UNIX systems.
The project also incorporates several key technologies for its infrastructure:
- Docker: For containerizing applications and ensuring consistent environments across different systems.
- Docker-compose: Manages multi-container Docker applications.
- Redis: An in-memory data structure store, used as a database and cache.
- PostgreSQL: A powerful, open-source object-relational database system.
- XNECT: Real-time motion capture software using standard video cameras.
The XNECT-Files are not included in this repository, because their license forbids it. You have to manually request them from their page. The rest of the software is under the MIT license.
The app is available in my other repository, multipose-service-app.
The Android App in Action looks like this:
A demo video can be watched on YouTube (https://youtu.be/AUAMqcTYfvY)
You need a graphics card with CUDA-Support for the calculations.
This repository has been fully tested on Ubuntu 20.04. It might also work on other Environments, but you may need different installation steps to make it run.
Permission is hereby granted, free of charge, to any person or company obtaining a copy of this software and associated documentation files (the "Software") from the copyright holders to use the Software for any non-commercial purpose. Publication, redistribution and (re)selling of the software, of modifications, extensions, and derivates of it, and of other software containing portions of the licensed Software, are not permitted. The Copyright holder is permitted to publically disclose and advertise the use of the software by any licensee.
Packaging or distributing parts or whole of the provided software (including code, models and data) as is or as part of other software is prohibited. Commercial use of parts or whole of the provided software (including code, models and data) is strictly prohibited. Using the provided software for promotion of a commercial entity or product, or in any other manner which directly or indirectly results in commercial gains is strictly prohibited.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
However, in this repository the source code is automatically built. For that, please put the downloaded XNECT-Source code into services/xnect/xnect_library.
Please refrain from using the Software commercially, if you use XNECT or create a fork and use another multiperson-Framework.
First of all, you should install an NVIDIA driver with support for CUDA >10.0. It is recommended to get the newest driver from the NVIDIA site and install it. In Ubuntu, you can install the driver using Software Update, but this didn't work for me. You don't need to install the CUDA toolkit yet, the Docker Image will take care of that (more info on this page).
This image is heavily relying on the benefits of the software Docker. You must install it on your local machine.
Unfortunately, you have to do an easy manual extension of the docker-compose-ecosystem.
This step is very important, because otherwise, the docker-compose commands will not be available for this repository.
It is a widely known issue, that the NVIDIA toolkit is not integrated into docker-compose files yet.
Follow the steps of Installation from the repository https://github.com/NVIDIA/nvidia-container-runtime to make it work.
The openpose images come with docker, so you don't have to take care of that. Nonetheless, you have to download the right models for 3d evaluation and put them into the 3d_models folder.
The models are not mine, they are from the following repository: https://github.com/KevinLTT/video2bvh. They can be downloaded from Google Drive.
You only have to download the openpose_video_pose_243f folder and extract the files best_58.58.pth and video_pose.yaml
directly into the services/web/3d_models folder.
From docker-compose.yml:
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=CHANGE_THIS_TO_A_STRONG_PASSWORD
- POSTGRES_DB=mocap
From .env.dev:
DATABASE_URL=postgresql://user:CHANGE_THIS_TO_A_STRONG_PASSWORD@db:5432/mocap
From .env.dev:
SECRET_KEY=CHANGE_THIS_KEY
The Server Settings can be accessed in file services/web/project/config.py. The Reverse Proxy can be configured using file services/nginx/nginx.conf
Running this project is fairly easy. You just have to type docker-compose build to build the image and docker-compose up
to run the application. Docker will take care of creating and managing the containers and installing all the requirements.
If an error occurs, it is probably due to a wrong NVIDIA-CUDA-Docker configuration.
Please refer to docker-compose Extension to fix that error.
The debugged Server can be accessed via localhost:80
If you want to run the project in debug mode you have to type docker-compose -f docker-compose-dev.yml.
The flask server can be accessed via localhost:5000
If everything works, your API will be accessible at 127.0.0.1 (localhost). All methods are descripted using a swagger documentation.
The Methods are also listed in the wiki.
- video2bvh by KevinLTT - MIT License
- BVHSmooth by Virgínia Balbo - MIT License
- XNECT by Mehta et al. (Source files not included in this repository) - ONLY FOR NON-COMMERCIAL PURPOSES
Frameworks used can be found in the according requirements.txt (Managed by pip package manager)