📣 Publication on Nature Methods: https://www.nature.com/articles/s41592-024-02370-y. Free access: https://rdcu.be/dQuw7
👇 Want to Try the Chatbot? Visit here!
Welcome to the BioImage.IO Chatbot user guide. This guide will help you get the most out of the chatbot, providing detailed information on how to interact with it and retrieve valuable insights related to computational bioimaging.
The BioImage.IO Chatbot is a versatile conversational agent designed to assist users in accessing information related to computational bioimaging. It leverages the power of Large Language Models (LLMs) and integrates user-specific data to provide contextually accurate and personalized responses. Whether you're a researcher, developer, or scientist, the chatbot is here to make your bioimaging journey smoother and more informative.
You can watch a demonstration of the BioImage.IO Chatbot performing image analysis in this video.
The following diagram shows how the chatbot works:
The BioImage.IO Chatbot is equipped with an array of capabilities designed to enhance the bioimaging experience:
-
Contextual and Personalized Response: Interprets the context of inquiries to deliver relevant and accurate responses. Adapts interactions based on user-specific background information to provide customized advice.
-
Comprehensive Data Source Integration: Accesses a broad range of databases and documentation for bioimaging, including bio.tools, ImageJ.net, deepImageJ, ImJoy, and bioimage.io. Details on the supported sources are maintained in the
knowledge-base-manifest.yamlfile. -
Advanced Query Capabilities: Generates and executes Python scripts for detailed queries within structured databases such as CSV, JSON files, or SQL databases, facilitating complex data retrievals.
-
AI-Powered Analysis and Code Interpretation: Directly runs complex image analysis tasks using advanced AI models like Cellpose, via an embedded code interpreter.
-
Performance Enhancements with ReAct and RAG: Utilizes a Retrieval Augmented Generation system with a ReAct loop for dynamic, iterative reasoning and tool engagement, improving response quality.
-
Extension Mechanism for Developers: Allows for the development of custom extensions using ImJoy plugins or hypha services within Jupyter notebooks, enhancing flexibility and integration possibilities.
-
Vision Inspection and Hardware Control: Features a Vision Inspector extension powered by GPT-4 for visual feedback on image content and analysis outcomes, and demonstrates potential for controlling microscopy hardware in smart microscopy setups.
-
Interactive User Interface and Documentation: Offers a user-friendly interface with comprehensive support documents, ensuring easy access to its features and maximizing user engagement.
We are providing a public chatbot service for you to try out. You can access the chatbot here.
Please note that the chatbot is still in beta and is being actively developed, we will log the message you input into the chatbot for further investigation of issues and support our development. See the Disclaimer for BioImage.IO Chatbot. If you want to to remove your chat logs, please contact us via this form.
Here you can find usage guide and more examples: Usage guide and example screenshots.
If you encounter any issues, please report them via Github.
To ask the chatbot a question, type your query and send it. The chatbot will analyze your question and provide a relevant response. You can ask questions related to bioimaging, software tools, models, and more.
The chatbot uses your user profile information, such as your name, occupation, and background, to personalize its responses. This ensures that the information you receive is tailored to your specific needs.
You can also set up your own chatbot server. Please refer to the installation guide for detailed instructions on how to set up the chatbot server on your local machine or server.
Please read the technical overview for more details about the chatbot's design and implementation.
The BioImage.IO Chatbot is designed to be extensible, allowing developers to create custom extensions to add new functionalities to the chatbot. You can create extensions to integrate new tools, databases, and services into the chatbot, making it more powerful and versatile. See the development guide for more details.
The BioImage.IO Chatbot is a community-driven project. We welcome contributions from the community to help improve the chatbot's knowledge base and make it more informative and useful to the community.
For more information, please visit the contribution guidelines.
If you are a tool developer or a database maintainer related to bioimaging, you can join us as a community partner. Please get in touch with us via Github issues.
If you have any questions, need assistance, or want to contribute to the chatbot's knowledge base, please do not hesitate to contact us via Github issues. Our team is here to help you get started and make valuable contributions.
Thanks for your support and helping make the BioImage.IO Chatbot more informative and useful to the community.
For detailed description of our work, please read our preprint: 
To reproduce the use cases described in Figure 2 in the manuscript, please refer to the reproducing example usage scenarios.
If you use the BioImage.IO Chatbot in your research, please cite us:
Lei, W., Fuster-Barceló, C., Reder, G. et al. BioImage.IO Chatbot: a community-driven AI assistant for integrative computational bioimaging. Nat Methods 21, 1368–1370 (2024). https://doi.org/10.1038/s41592-024-02370-y
The BioImage.IO Chatbot is greatly enriched by the valuable resources contributed by the global bioimage analysis community. We extend our deepest gratitude to all authors and contributors listed in the knowledge base manifest, especially the original authors of the materials integrated into our extensions. Their work has significantly enhanced the capabilities of this chatbot, providing users with high-quality information and tools.
We would also like to thank the open-source community for their ongoing support and innovation, which continuously drives the development and improvement of the BioImage.IO platform.
We thank AI4Life consortium for its crucial support in the development of the BioImage.IO Chatbot.
AI4Life has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101057970. Views and opinions expressed are, however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
Thank you for supporting the BioImage.IO project and contributing to the advancement of bioimage analysis!
These resources are primarily utilized for research purposes in compliance with the European Union's copyright exception on Text and Data Mining (TDM) as outlined in Directive (EU) 2019/790.
The full list of sources and resources used by the BioImage.IO Chatbot can be found in our knowledge base manifest. We kindly ask users to carefully verify the accuracy and completeness of information with the original sources. For any work derived from these resources, please ensure that you respect the respective licenses and adhere to the conditions set forth by the original authors.
If you are an author of any material used within the BioImage.IO Chatbot and wish to have your content removed, please contact us directly. We are committed to respecting the rights of all contributors and will promptly address your request.
See the full Disclaimer for BioImage.IO Chatbot.