An Unreal Engine set of tools to build Embodied Conversational Agents using easy, Blueprint based, interfaces towards powerful AI technologies.
Forum thread
·
Youtube channel
·
Facebook page
FANTASIA is a collection of tools for UE that we have developed over the years to work with Embodied Conversational Agents at the University of Padua and at the University of Naples Federico II in Italy. Since we have been working with different characters for different tasks, I developed these tools so that the functionalities characterizing one agent could be easily transferred to others. Since we have done this multiple times with minimal effort, now, and since there appears to be an increasing interest towards Virtual Humans in UE, we decided to take all the dirty code we have and re-engineer it so that we could distribute it to the community. We hope it will be helpful for other colleagues in the academy but we also hope it will help people, in general, produce amazing stuff.
- Clone or download the repository.
- If not already present, create a folder named "Plugins" in your PROJECT
- Copy the contents of the FANTASIA folder from the repository to the Plugins folder
- Launch UE. You should see FANTASIA listed in the Plugins list
If you get an error stating that your project cannot be launched because of FANTASIA, please check that the dll files included in the ThirdParty->Redist folder of the repository have been copied where your project's binaries are found. These should be under Binaries->Win64. If the dll files are not there, please copy them manually. FANTASIA currently uses dynamically linked libraries from Microsoft and Amazon to access services exposed to Blueprints.
If you use FANTASIA for your research, please cite the following papers in your publications
Origlia, A., Cutugno, F., Rodà, A., Cosi, P., & Zmarich, C. (2019). FANTASIA: a framework for advanced natural tools and applications in social, interactive approaches. Multimedia Tools and Applications, 78(10), 13613-13648.
Origlia, A., Di Bratto, M., Di Maro, M., & Mennella, S. (2022). Developing Embodied Conversational Agents in the Unreal Engine: The FANTASIA Plugin. In Proceedings of the 30th ACM International Conference on Multimedia (pp. 6950-6951).
SPECIAL - Supporting Patients with Embodied Conversational Interfaces and Argumentative Language (funded by the University of Naples Federico II): developing a Virtual Human to increase health literacy and fight stereotypes about epilepsy.
VISIT3D - Logogramma S.r.l. in collaboration with URBAN/ECO Research Center (University of Naples Federico II): developing a Virtual Reality guide for reflection about visual arts.
Antonio Origlia - antonio.origlia [at] unina.it
Project Link: https://github.com/antori82/FANTASIA
Unreal Engine Forum discussion page
FANTASIA would not be the same without the precious contribution from the Linguistics field provided by
- Dr. Maria Di Maro - (PhD) Common ground management and Clarification Requests generation
- Martina Di Bratto - (PhD Student) Argumentative Dialogue Management
Special thanks to the students who contributed to FANTASIA
- Antimo De Biase - Neo4j component prototype development
- Fabrizio Musella - aGrUM integration prototype development
- Roberto Basile Giannini - General improvements and aGrUM updates (Influence Diagrams integration)
- Marco Galantino - TTS General interface
- Yegor Napadystyy - Audio2Face prototype connector