-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LeafSnap30 model #567
Open
elboyran
wants to merge
43
commits into
onnx:main
Choose a base branch
from
elboyran:leafsnap30-model
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
LeafSnap30 model #567
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
715fd51
Update README.md
elboyran fe3738a
Create README.md
elboyran 90617d5
Update README.md
elboyran 9cdb00e
Create test.md
elboyran acc19e5
Delete test.md
elboyran 888bee0
Update README.md
elboyran 98b7d4c
Update README.md
elboyran 9b44c80
Merge branch 'onnx:main' into leafsnap30-model
elboyran 2573bf7
Merge branch 'onnx:main' into elboyran-leafsnap
elboyran f35c75f
Update README.md
elboyran d6dafa8
Update README.md
elboyran f1a09ba
Update README.md
elboyran 26c7d02
Folder for the model
elboyran 46b5290
Merge pull request #1 from elboyran/elboyran-leafsnap
elboyran 805671a
Create leafsnap_model.onnx
elboyran 4fb39f6
Delete README.md
elboyran 46eb4dd
Update README.md
elboyran dcfbfc3
Update vision/classification/leafsnap30/README.md
elboyran 265fbb5
Update README.md
elboyran 7a8dd90
Merge pull request #2 from elboyran/elboyran-model-upload
elboyran c399321
Update README.md
elboyran f765175
Update README.md
elboyran 1dcb2c2
Merge pull request #3 from elboyran/elboyran-model-inference
elboyran 8e74d3a
Merge branch 'onnx:main' into leafsnap30-model
elboyran b13cbee
Update README.md
elboyran 4ed0aac
Merge pull request #4 from elboyran/elboyran-references
elboyran 100ed82
Update README.md
elboyran bccb7fb
Merge pull request #6 from elboyran/elboyran-references
elboyran 1d6b9f7
Add model section
loostrum 7612a9a
Fix train accuracy
loostrum e6892fb
Fix URL
loostrum 4c400f2
Fix accuracies
loostrum be570f6
Update validation accuracy section
loostrum 7c93951
Merge pull request #7 from elboyran/add_model_creation_section
elboyran 773cdd3
Add test data for leafsnap30
loostrum 75211bf
Merge branch 'leafsnap30-model' into add_test_data_creation_section
loostrum 6472d82
Remove test data creation instructions from leafsnap README
loostrum 4b38c66
Update leafsnap test file
loostrum 57be194
Add link to leafsnap30 test data to README
loostrum 755b3b2
Add LeafSnap30 to ONNX Hub json
loostrum 43d083d
Merge pull request #8 from elboyran/add_test_data_creation_section
elboyran 1b3ac16
Merge branch 'onnx:main' into leafsnap30-model
elboyran 7a7e9b1
Merge branch 'main' into leafsnap30-model
jcwchen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# LeafSnap30 | ||
|
||
## Description | ||
LeafSnap30 is a Neural Network model trained on the [LeafSnap 30 dataset](https://zenodo.org/record/5061353/). It addresses an image classificaiton task- identifying 30 tree species from images of their leaves. This task has been approached with classical computer vision methods a decade ago on more species dataset containing artifacts, while this model is trained on a smaller and cleaner dataset, particularly useful for demonstrating NN classification on simple, yet realistic scientific task. | ||
|
||
## Model | ||
|
||
|Model |Download | Download (with sample test data)|ONNX version|Opset version|Accuracy | | ||
|-------------|:--------------|:--------------|:--------------|:--------------|:--------------| | ||
|Model Name | Relative link to ONNX Model with size | tar file containing ONNX model and synthetic test data (in .pb format)|ONNX version used for conversion| Opset version used for conversion|Accuracy values | | ||
|LeafSnap30| [1.48 MB](model/leafsnap_model.onnx) | [1.55 MB](model/leafsnap30.tar.gz) | 1.9.0 |11 | train: 95%, validation: 86, test: 83% | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would suggest using leafsnap-11.onnx as the model name (to represent the used opset_version) and please use consistent name for the .tar.gz file like leafsnap-11.tar.gz. |
||
|
||
### Source | ||
Pytorch LeafSnap30 ==> ONNX LeafSnap30 | ||
|
||
## Inference | ||
The steps needed to run the pretrained model with the onnxruntime are implemented within the explainability library [dianna](https://github.com/dianna-ai/dianna) in this [code](https://github.com/dianna-ai/dianna/blob/main/dianna/utils/onnx_runner.py). An example [tutorial notebook](https://github.com/dianna-ai/dianna/blob/main/tutorials/lime_images.ipynb) shows on how to use the model with dianna. | ||
|
||
### Input | ||
The input to the model is a ``float32`` tensor of shape ``(-1, 3, 128, 128)``, where -1 is the batch axis. Each image is a ``128x128`` RGB image, with the colour channels as first axis. | ||
|
||
### Preprocessing | ||
The input image is loaded to a numpy array. The pixel values are then scaled to the 0-1 range. | ||
|
||
Example: | ||
|
||
``` | ||
# load and plot the example image | ||
img = np.array(Image.open(f'data/leafsnap_example_{true_species}.jpg')) | ||
|
||
plt.imshow(img) | ||
plt.title(f'Species: {true_species}'); | ||
|
||
# the model expects float32 values in the 0-1 range for each pixel, with the colour channels as first axis | ||
# the .jpg file has 0-255 ints with the channel axis last so it needs to be changed | ||
input_data = img.transpose(2, 0, 1).astype(np.float32) / 255. | ||
``` | ||
|
||
### Output | ||
Output of this model is the likelihood of each tree species before softmax, a tensor of shape ``` 1 x 30```. | ||
|
||
## Model Creation | ||
|
||
### Dataset (Train and validation) | ||
From the original LeafSnap dataset, the 30 most prominent classes were selected. The images taken in a lab were cropped semi-manually to remove any rulers and color calibration image parts. Notebooks describing these steps are available [here](https://github.com/dianna-ai/dianna-exploration/tree/main/example_data/dataset_preparation/LeafSnap). The LeafSnap30 dataset is also available on [Zenodo](https://zenodo.org/record/5061353). | ||
|
||
### Training | ||
The model is a CNN with 4 hidden layers, built in PyTorch and converted to ONNX. A notebook for the generation of the model, including the used hyperparameters, is available [here](https://github.com/dianna-ai/dianna-exploration/main/example_data/model_generation/). | ||
|
||
### Validation accuracy | ||
The notebook used for training the model also shows how accuracy on the validation and test datasets is calculated. The actual values were taken from the hyperparameter sweep executed with [Weights & Biases](wandb.ai). | ||
|
||
### References | ||
[Leafsnap: A Computer Vision System for Automatic Plant Species Identification](https://rdcu.be/c0aBX) (original LeafSnap paper) | ||
|
||
[LeafSnap30](https://zenodo.org/record/5061353/) (Zenodo dataset archive) | ||
|
||
DIANNA: Deep Insight And Neural Network Analysis [![status](https://joss.theoj.org/papers/f0592c1aecb3711e068b58970588f185/status.svg)](https://joss.theoj.org/papers/f0592c1aecb3711e068b58970588f185) | ||
|
||
## Contributors | ||
- [Leon Oostrum](https://github.com/loostrum) (Netherlands eScience Center) | ||
- [Christiaan Meijer](https://github.com/cwmeijer) (Netherlands eScience Center) | ||
- [Yang Liu](https://github.com/geek-yang) (Netherlands eScience Center) | ||
- [Patrick Bos](https://github.com/egpbos) (Netherlands eScience Center) | ||
- [Elena Ranguelova](https://github.com/elboyran) (Netherlands eScience Center) | ||
|
||
## License | ||
Apache 2.0 | ||
<hr> |
Git LFS file not shown
Git LFS file not shown
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not necessary?