Explore and experience running Large Language Models offline on your edge devices with the RWKV App.
RWKV App is an experimental application that brings Large Language Models (LLMs) directly to your Android/iOS devices. You can experiment with different models, engage in chats, generate speech, perform visual understanding, and more! All computations are performed locally, and no internet connection is required after loading the model.
Overview
The RWKV App supports multi-turn conversations, text-to-speech, visual understanding, and various other tasks.
- 📱 Run Locally, Fully Offline: Experience the magic of generative AI without an internet connection. All processing is done directly on your device.
- 🤖 Switch Models Freely: Easily download and switch between different models from Hugging Face to compare their performance.
- 💬 AI Chat: Engage in fluent multi-turn conversations.
- 🔊 Text-to-Speech (TTS): Convert text into natural-sounding speech.
- 🖼️ Visual Understanding: Explore image-based AI use cases.
- 🌓 Dark Mode: Supports comfortable use in various lighting conditions.
| RWKV Chat | RWKV See | RWKV Talk | RWKV Sudoku | RWKV Othello | |
|---|---|---|---|---|---|
| Android APK Download Link | huggingface / pgyer | huggingface / pgyer | huggingface / pgyer | huggingface / pgyer | huggingface / pgyer |
| iOS | testflight | testflight | testflight | - | testflight |
| Windows | QQ Group / Discord | ||||
| macOS | QQ Group / Discord | ||||
Note
In the future, we will integrate all separate features into the RWKV Chat app to provide a unified experience.
When you first open the app, a model selection panel will appear. Please choose the model weights you want to use based on your needs.
Warning
Devices older than the iPhone 14 may not be able to smoothly run models with 1.5B / 2.9B parameters.
- Clone the repository:
git clone https://github.com/MollySophia/rwkv_mobile_flutter.git
# Make sure the rwkv_mobile_flutter and RWKV_APP are in the same directory
git clone https://github.com/RWKV-APP/RWKV_APP.git
cd RWKV_APP- Install dependencies:
flutter pub get- Run the application:
flutter run- Flutter: An open-source framework for building cross-platform user interfaces, supporting Android, iOS, Windows, and macOS.
- Dart FFI (Foreign Function Interface): Used for efficient communication between Dart and the C++ inference engine.
- C++ Inference Engine: The core on-device inference engine, built with C++, supporting multiple model formats and hardware acceleration (CPU/GPU/NPU).
- Hugging Face: An open-source community providing models, datasets, and tools; used here as the source for model weights.
- UI Refactoring
- Integrate all features into the RWKV Chat app
- Support more model weights
- Support more hardware
- Support more operating systems
- Support more devices (e.g., watches, VR glasses)
This is an experimental early-stage version, and your feedback is crucial to us!
- 🐞 Found a bug or issue? Report it here!
- 💡 Have a suggestion? Suggest a feature!
This project is licensed under the Apache License 2.0. See the LICENSE file for details.
