Skip to content

Commit

Permalink
Update docs for the demo app. (#2921)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #2921
overriding_review_checks_triggers_an_audit_and_retroactive_review
Oncall Short Name: executorch

Differential Revision: D55885790

fbshipit-source-id: bb62a42b74ecdfb2e1f6bcebab979e2e8fcf0a3c
  • Loading branch information
shoumikhin authored and facebook-github-bot committed Apr 8, 2024
1 parent 3e256ff commit 9ba8bc9
Show file tree
Hide file tree
Showing 8 changed files with 46 additions and 5 deletions.
Binary file added docs/source/_static/img/llama_ios_app.mp4
Binary file not shown.
Binary file added docs/source/_static/img/llama_ios_app.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/source/demo-apps-ios.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
```{include} ../../examples/demo-apps/apple_ios/README.md
```{include} ../../examples/demo-apps/apple_ios/ExecuTorchDemo/README.md
2 changes: 1 addition & 1 deletion docs/source/llm/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -626,7 +626,7 @@ The rest of the steps will be the same as the normal flow. Now you can run this
## How to build Mobile Apps
You can also execute an LLM using ExecuTorch on iOS and Android

**For iOS details see the [iOS Sample App](https://github.com/pytorch/executorch/tree/main/examples/demo-apps/apple_ios).**
**For iOS see the [iLLaMA App](https://pytorch.org/executorch/main/llm/llama-demo-ios.html).**


**For Android see the [Android Instructions](https://pytorch.org/executorch/main/llm/llama-demo-android.html).**
2 changes: 2 additions & 0 deletions docs/source/llm/llama-demo-ios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
```{include} ../../../examples/demo-apps/apple_ios/LLaMA/README.md
```
File renamed without changes.
37 changes: 37 additions & 0 deletions examples/demo-apps/apple_ios/LLaMA/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Building ExecuTorch LLaMA iOS Demo App

This app demonstrates the use of the LLaMA chat app demonstrating local inference use case with ExecuTorch.

<img src="../_static/img/llama_ios_app.png" alt="iOS LLaMA App" /><br>

## Prerequisites
* [Xcode 15](https://developer.apple.com/xcode).
* [iOS 17 SDK](https://developer.apple.com/ios).
* Set up your ExecuTorch repo and environment if you haven’t done so by following the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) to set up the repo and dev environment.

## Exporting models
Please refer to the [ExecuTorch Llama2 docs](https://github.com/pytorch/executorch/blob/main/examples/models/llama2/README.md) to export the model.

## Run the App

1. Open the [project](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/apple_ios/LLaMA/LLaMA.xcodeproj) in Xcode.
2. Run the app (cmd+R).
3. In app UI pick a model and tokenizer to use, type a prompt and tap the arrow buton as on the [video](../_static/img/llama_ios_app.mp4).

```{note}
ExecuTorch runtime is distributed as a Swift package providing some .xcframework as prebuilt binary targets. Xcode will dowload and cache the package on the first run, which will take some time.
```

## Copy the model to Simulator

1. Drag&drop the model and tokenizer files onto the Simulator window and save them somewhere inside the iLLaMA folder.
2. Pick the files in the app dialog, type a prompt and click the arrow-up button.

## Copy the model to Device

1. Wire-connect the device and open the contents in Finder.
2. Navigate to the Files tab and drag&drop the model and tokenizer files onto the iLLaMA folder.
3. Wait until the files are copied.

## Reporting Issues
If you encountered any bugs or issues following this tutorial please file a bug/issue here on [Github](https://github.com/pytorch/executorch/issues/new).
8 changes: 5 additions & 3 deletions examples/models/llama2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,11 +218,13 @@ adb push cmake-out-android/examples/models/llama2/llama_main /data/local/tmp/lla
```
adb shell "cd /data/local/tmp/llama && ./llama_main --model_path <model.pte> --tokenizer_path <tokenizer.bin> --prompt "Once upon a time" --seq_len 120
```
## Step 6: Build iOS and/or Android apps
## Step 6: Build Mobile apps
TODO
### iOS
Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-demo-ios.html) to for full instructions on building the iOS LLAMA Demo App.
### Android app
### Android
Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-demo-android.html) to for full instructions on building the Android LLAMA Demo App.
# What is coming next?
Expand Down

1 comment on commit 9ba8bc9

@mergennachin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoumikhin

Please cherry pick using #2666

Please sign in to comment.