Contribute Guide on Deep Learning for Gesture Recognition on ESP32-S3 #350
Conversation
|
Thank you very much @BlakeHansen130 for your contribution! |
|
Thank you very much for starting the review process! I'll keep an eye out for any feedback and will be happy to make any changes necessary. |
| @@ -0,0 +1,382 @@ | |||
| --- | |||
| title: "Deep Learning for Gesture Recognition on ESP32-S3 from Training to Deployment" | |||
There was a problem hiding this comment.
I suggest changing the title to:
"Deep Learning for Gesture Recognition on ESP32-S3: From Training to Deployment"
| - ESP-DL | ||
| - Model Quantization | ||
| --- | ||
| Integrating deep learning capabilities into embedded systems has become a crucial aspect of modern IoT applications. Although powerful deep learning models can achieve high recognition accuracy, deploying these models on resource-constrained devices poses considerable challenges. This article presents a gesture recognition system based on the ESP32-S3, detailing the entire workflow from model training to deployment on embedded hardware. The complete project implementation and code are available at [gesture-recognition-model](https://github.com/BlakeHansen130/gesture-recognition-model). By utilizing ESP-DL(master branch) and incorporating efficient quantization strategies with ESP-PPQ, this study demonstrates the feasibility of achieving gesture recognition on resource-limited devices while maintaining satisfactory accuracy. Additionally, insights and methodologies were inspired by the work described in [Espressif's blog on hand gesture recognition](https://developer.espressif.com/blog/hand-gesture-recognition-on-esp32-s3-with-esp-deep-learning/), which significantly influenced the approach taken in this article. |
| include($ENV{IDF_PATH}/tools/cmake/project.cmake) | ||
| project(gesture_recognition) | ||
| ``` | ||
| __Note: Ensure that CMake can locate the esp-dl library cloned from GitHub by using relative paths to reference the esp-dl directory. |
| @@ -0,0 +1,382 @@ | |||
| --- | |||
There was a problem hiding this comment.
Could you add the featured image? If you have questions about the image, please ask here.
The file should be placed at the same level as this MD file and the name should contain features and the format MUST be Webp.
| @@ -0,0 +1,382 @@ | |||
| --- | |||
| title: "Deep Learning for Gesture Recognition on ESP32-S3 from Training to Deployment" | |||
| date: 2024-11-30 | |||
There was a problem hiding this comment.
The date should be updated to the expected publication date. We will ask you to change ass soon as this PR is approved.
| - ESP-DL | ||
| - Model Quantization | ||
| --- | ||
| Integrating deep learning capabilities into embedded systems has become a crucial aspect of modern IoT applications. Although powerful deep learning models can achieve high recognition accuracy, deploying these models on resource-constrained devices poses considerable challenges. This article presents a gesture recognition system based on the ESP32-S3, detailing the entire workflow from model training to deployment on embedded hardware. The complete project implementation and code are available at [gesture-recognition-model](https://github.com/BlakeHansen130/gesture-recognition-model). By utilizing ESP-DL(master branch) and incorporating efficient quantization strategies with ESP-PPQ, this study demonstrates the feasibility of achieving gesture recognition on resource-limited devices while maintaining satisfactory accuracy. Additionally, insights and methodologies were inspired by the work described in [Espressif's blog on hand gesture recognition](https://developer.espressif.com/blog/hand-gesture-recognition-on-esp32-s3-with-esp-deep-learning/), which significantly influenced the approach taken in this article. |
There was a problem hiding this comment.
Add a link to ESP-DL(master branch).
[ESP-DL](https://github.com/espressif/esp-dl)
| - ESP-DL | ||
| - Model Quantization | ||
| --- | ||
| Integrating deep learning capabilities into embedded systems has become a crucial aspect of modern IoT applications. Although powerful deep learning models can achieve high recognition accuracy, deploying these models on resource-constrained devices poses considerable challenges. This article presents a gesture recognition system based on the ESP32-S3, detailing the entire workflow from model training to deployment on embedded hardware. The complete project implementation and code are available at [gesture-recognition-model](https://github.com/BlakeHansen130/gesture-recognition-model). By utilizing ESP-DL(master branch) and incorporating efficient quantization strategies with ESP-PPQ, this study demonstrates the feasibility of achieving gesture recognition on resource-limited devices while maintaining satisfactory accuracy. Additionally, insights and methodologies were inspired by the work described in [Espressif's blog on hand gesture recognition](https://developer.espressif.com/blog/hand-gesture-recognition-on-esp32-s3-with-esp-deep-learning/), which significantly influenced the approach taken in this article. |
|
|
||
| The development process requires two distinct Conda environments to handle different stages of the workflow. The primary training environment, designated as 'dl_env', manages dataset preprocessing, model training, and basic evaluation tasks. A separate quantization environment, 'esp-dl', is specifically configured for model quantization, accuracy assessment, and ESP-DL format conversion. | ||
|
|
||
| For the deployment phase, ESP-IDF version 5.x is used, with specific testing conducted on v5.3.1. The implementation relies on the master branch of ESP-DL and ESP-PPQ for enhanced quantization capabilities. The specific versions used in this implementation can be obtained through: |
There was a problem hiding this comment.
To avoid confusion, please add this way:
The ESP-IDF version 5.3.1 is used for the deployment and testing phase.
| @@ -0,0 +1,382 @@ | |||
| --- | |||
There was a problem hiding this comment.
The article folder should be placed in the following structure:
└── content
└── blog
└── 2024
└── 12
So, move the article folder inside the blog/2024/12/ folder.
…-from-Training-to-Deployment/index.md to content/blog/2024/12/Deep-Learning-for-Gesture-Recognition-on-ESP32-S3-from-Training-to-Deployment/index.md
…ESP32-S3-from-Training-to-Deployment/features.webp
|
Thanks for the suggested fixes! I've updated the guide according to the review comments. Ready to merge after your approval. |
|
🎉 A preview for this PR is available at: https://preview-developer.espressif.com/pr350/ |
|
Sorry @pedrominatel, I accidentally clicked the refresh button on your review. I've already updated the date as well. |
|
|
||
| ## Experimental Results | ||
|
|
||
| The gesture recognition system on ESP32-S3 showed strong performance across multiple metrics. Both quantitative and qualitative evaluations were conducted. |
There was a problem hiding this comment.
If multiple metrics were mentioned here, they should also be specified somewhere in the text, along with what can be considered as strong performance.
Waiting for pictures to be added
Modified the article description of the quantitative part
…ESP32-S3-from-Training-to-Deployment/img/1
…ESP32-S3-from-Training-to-Deployment/features.webp to content/blog/2024/12/Deep-Learning-for-Gesture-Recognition-on-ESP32-S3-from-Training-to-Deployment/img/features.webp
Uploaded some pictures
…tion_quantization.webp
Inserted some pictures
Added article flow chart
|
Thank you for your review and valuable comments, @horw. I have addressed your feedback with the following updates:
Please let me know if there’s anything further I can clarify or improve. Thank you again for your insightful feedback! |
|
🎉 A preview for this PR is available at: https://preview-developer.espressif.com/pr350/ |
…ESP32-S3-from-Training-to-Deployment/img/features.webp to content/blog/2024/12/Deep-Learning-for-Gesture-Recognition-on-ESP32-S3-from-Training-to-Deployment/features.webp
|
🎉 A preview for this PR is available at: https://preview-developer.espressif.com/pr350/ |
|
@BlakeHansen130 Thank you for the update. Overall, it is better than the previous version. However, for your further contributions, I would suggest the following improvements in my opinion:
Consider whether your article offers enough information to effectively help readers solve the problem. Consider how your code snippets and images can help. When your article contains enough information to solve the related problem and helps me fully understand the steps you took and why, it will be a very valuable article. P.S. In the ML field, I like how Andrej Karpathy explains things. For example, his explanation of transformer models. Or 3Blue1Brown |
|
The folder name (URL slug) looks long, suggest shortening it from |
|
Hi @BlakeHansen130 , I am Kirill - a technical writer at Espressif. I do not possess sufficient technical knowledge in your particular area, but I have some experience structuring knowledge, shaping content, and presenting it to readers, both in technical documentation and articles for developers. I have read through your article and checked your project's repo. From what I understand, your article describes the methodology that you used to implement your project and gives some insights into the steps you took and techniques you employed. However, the purpose of your article is still not very clear. Let me explain what I mean. You state clearly the following:
I see that you provide an overview of the development process. However, you cover only certain sides of the development process and other sides are only briefly touched on with phrases, such as Usually before you write content, you decide on your target audience and the goal you want to achieve. I can see that your article might have the following potential purposes:
Let's quickly go through all the purposes and how they compare against your current state of the article:
Now after this analysis let's get back to your article. Can you please tell me what target audience and what your goal is?
Now please let me know what you think. |
|
@f-hollow Thank you for the feedback! During my undergraduate design project, I was inspired by Bukharai's gesture recognition tutorial. After successfully implementing the complete workflow with newer versions of esp-idf, esp-dl and esp-ppq (as discussed in alibukharai/Blogs#10), I wanted to share this article to help others exploring similar implementations. |
…s in the article, added some experimental data and model visualization picture
|
@horw Thank you for your feedback. Here are my implemented changes:
Regarding the structure - I've maintained it as is because it directly mirrors my project's implementation workflow. Each section progresses naturally from data preprocessing through model development and quantization to deployment, just like the organized hierarchy in my code repository. This parallel between article structure and actual development process helps readers understand both the theoretical concepts and practical implementation steps simultaneously. I'm also a fan of 3Blue1Brown's excellent teaching style and continue learning from his videos, and I'm still working toward that level of clarity in my explanations. |
|
🎉 A preview for this PR is available at: https://preview-developer.espressif.com/pr350/ |
Description
Hello Espressif team,
This PR adds an article discussing the implementation of deep learning-based gesture recognition on the ESP32-S3, covering the journey from training to deployment. The goal of this article is to provide developers with a complete guide, helping them understand:
I believe this article could serve as a valuable resource for developers looking to apply machine learning to embedded systems, and I'm eager to contribute to the Espressif community.
Related
This contribution is related to the discussion in issue #344. Please feel free to refer to it for more context.
Testing
The article has undergone thorough grammar and content reviews to ensure clarity and accuracy. I have also tested the example code provided in the guide and verified that the steps work properly in a typical setup.
Checklist
Please let me know if there are any areas where further improvements are needed or if there is additional information I can provide to make this article more helpful. I am open to any feedback and happy to make modifications as required.
Thank you very much for your time and consideration.
Best regards,
BlakeHansen130