electron-rare/esp-box
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'docs/fix_broken_reference' into 'master'

Browse Source 
docs: fix broken reference url

See merge request ae_group/esp-box!16
This commit is contained in:
Wu Jian Gang
2021-12-09 07:15:36 +00:00
parent 020cdd5681 fd9c3d66a3
commit 2514d7316e
4 changed files with 16 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/hardware_overview.md
+2
View File
@@ -1,3 +1,5 @@
* [中文版本](./hardware_overview_cn.md)
## Hardware Overview
![esp_box_hardware](_static/esp32_s3_box_hardware.svg)
docs/hardware_overview_cn.md
+2
View File
@@ -1,3 +1,5 @@
* [English Version](./hardware_overview.md)
## 硬件总览
![esp_box_hardware](_static/esp32_s3_box_hardware.svg)
docs/technical_architecture.md
+8 -8
View File
@@ -10,23 +10,23 @@ The ESP-BOX development framework includes three main layers: system layer, fram
### System Layer
ESP-IDF is the open-source development framework for Espressif SoCs including ESP32、ESP32-S and ESP32-C series. It integrates a large number of system-level basic components, a set of development tools for code compilation and debugging, and detailed development guidance documents:
ESP-IDF is the open-source development framework for Espressif SoCs including ESP32、ESP32-S and ESP32-C series. It integrates a large number of system-level basic components, a set of development tools for code compilation and debugging, and detailed development guidance documents:
1. System-level basic components, mainly including multi-tasking, multi-core support based on FreeRTOS, and memory heap allocator with external expansion of PSRAM; a uniform operation interface support for LL, HAL, Driver, and VFS layers of multiple ESP chips; multiple standard network protocol stacks like TCP/IP, HTTP, MQTT, WebSocket, etc.
2. Set of development tools, mainly including GCC cross-toolchain for code building, JTAG debugging tool based on OpenOCD, real-time tracking based on Segger SystemView, Flash and eFuse programmers, etc.
3. Development guidance documents, API references, guidelines, and notes of Espressif’s various chip platforms and software versions. Developers can view these documents [Online](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/) or [Download](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/esp-idf-en-v4.4-dev-3540-g4e03a9c-esp32.pdf).
3. Development guidance documents, API references, guidelines, and notes of Espressif’s various chip platforms and software versions. Developers can view these documents [Online](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/).
### Framework Layer
In addition to system-level support, ESP-BOX also integrates several excellent solution frameworks for typical problems in different scenarios. Including the speech recognition framework ESP-SR, the human-computer interaction framework ESP-HMI, and the end-to-end IoT development framework ESP-RainMaker. The functions and customization methods of these solution frameworks will be briefly introduced as follows:
In addition to system-level support, ESP-BOX also integrates several excellent solution frameworks for typical problems in different scenarios. Including the speech recognition framework ESP-SR, the human-computer interaction framework ESP-HMI, and the end-to-end IoT development framework ESP-RainMaker. The functions and customization methods of these solution frameworks will be briefly introduced as follows:
**ESP-SR**
ESP-SR is a speech recognition framework developed by Espressif for the field of intelligent speech. ESP-SR can provide high-performance and high-reliability speech recognition functions for developers by using ESP32-S3 AI extension instructions. Now, this framework has three modules:
1. Espressif's acoustic algorithm ([Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/master/acoustic_algorithm)) , which is specially designed to improve speech recognition performance in far-field or noisy environments to enable developers to obtain high-quality and stable audio data. It mainly includes AEC (Acoustic Echo Cancellation) , BSS (Blind Source Separation), NS (Noise Suppression). ESP-SR encapsulates the above algorithms into simple APIs. Without undersatanding details of these algorithms, developers only need to arrange and combine the algorithms that need to be used for specific application scenarios, and input the voice data to be processed according to the API format requirements and can get the results.
2. Espressif wake word engine ([WakeNet](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md)), which is used to monitor specific language fragments in real-time in a continuous audio stream, and wake up the smart device from the standby state to the voice interaction state. WakeNet uses MFC (Mel-Frequency Cepstrum) to obtain the features of the input audio clip , and classifies the characteristic signals by the neural network algorithm optimized for ESP32-S3. It can trigger the wake-up signal when monitor the command word, and the [Effective recognition rate](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md#performance-test) is not less than 80% in noisy environment. ESP-BOX provides different offline wake words like “Xiao Ai Tong Xue” “Hi, Lexin” etc. and also allows [customized wake word](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/ESP_Wake_Words_Customization.md).
3. Espressif wake-up word engine ([WakeNet](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md)), which is used to monitor specific language fragments in real-time in a continuous audio stream, and wake up the smart device from the standby state to the voice interaction state. WakeNet uses MFC (Mel-frequency Cepstrum) to obtain the features of the input audio clip, then classifies the characteristic signals by the neural network algorithm optimized for ESP32-S3. That can trigger the wake-up signal when analyzed the command word, and the [Effective recognition rate](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md#performance-test) is more than 80% in a noisy environment. ESP-BOX provides different offline wake words like “Hi, ESP” and “Hi, Lexin”. And also allows [customized wake word](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/ESP_Wake_Words_Customization.md).
1. Espressif's acoustic algorithm ([Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/acoustic_algorithm)), which is specially designed to improve speech recognition performance in far-field or noisy environments to enable developers to obtain high-quality and stable audio data. It mainly includes AEC (Acoustic Echo Cancellation), BSS (Blind Source Separation), NS (Noise Suppression). ESP-SR encapsulates the above algorithms into simple APIs. Without undersatanding details of these algorithms, developers only need to arrange and combine the algorithms that need to be used for specific application scenarios, and input the voice data to be processed according to the API format requirements and can get the results.
2. Espressif wake-up word engine ([WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)), which is used to monitor specific language fragments in real-time in a continuous audio streaming, and wake up the smart device from the standby state to the voice interaction state. WakeNet uses MFC (Mel-frequency Cepstrum) to obtain the features of the input audio clip, then classifies the characteristic signals by the neural network algorithm optimized for ESP32-S3. That can trigger the wake-up signal when analyzed the command word, and the [Effective recognition rate](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test) is more than 80% in a noisy environment. ESP-BOX provides different offline wake words like "Hi, ESP" or "Hi, Lexin". And also allows [customized wake word](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/ESP_Wake_Words_Customization.md).
3. Espressif multi-command recognization model ([MultiNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md)), which is designed to provide a flexible command recognition after the device is awakened. MultiNet supports both Chinese and English speech commands, like "Guan Bi Dian Deng", "Turn on the air conditioner" through a lightweight model specially designed based on CRNN and CTC, and supports up to 200 offline customized speech commands. The command words can be added and modified simply through configuring [Chinese Pinyin string](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) or [English Phonetic String](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) and the corresponding ID. Developers do not need to understand the details of the MultiNet nor re-train the model. the event will be triggered after the command words are recognized.
**ESP-HMI**
@@ -42,11 +42,11 @@ ESP-HMI is a human-computer interaction solution designed by Espressif based on
1. Device-side SDK, developers can use Espressif chips to define their IoT products by designing device parameters and attributes, and complete device-side firmware development quickly.
2. Secure cloud middleware, developers do not need to manage device certificates and cloud infrastructure. They can directly use Espressif chips to quickly establish a secure communication link with the cloud.
3. iOS / Android APP, which integrates the necessary functions such as device discovery, Wi-Fi provisioning, user login, device association and control, and developers do not need to carry out additional code development work.
3. iOS / Android APP, which integrates the necessary functions such as device discovery, Wi-Fi provisioning, user login, device association, and control. Developers do not need to carry out additional code development work.
**AI Instructions and Libraries**
With the help of ESP32-S3 Xtensa® 32bit LX7 dual-core CPU and AI / DSP extended instruction set, ESP32-S3 can efficiently complete vector operations, complex number operations, FFT and other operations, which can accelerate neural network calculations and digital signal processing calculation capabilities. AI developers can implement high-performance image recognition, voice wake-up and recognition with Espressif's open source software library based on the AI / DSP extended instruction set.
With the help of ESP32-S3 Xtensa® 32bit LX7 dual-core CPU and AI / DSP extended instruction set, ESP32-S3 can efficiently complete vector operations, complex number operations, FFT, and other operations, which can accelerate neural network calculations and digital signal processing calculation capabilities. AI developers can implement high-performance image recognition, voice wake-up, and recognition with Espressif's open-source software library based on the AI / DSP extended instruction set.
**Device Drivers**
docs/technical_architecture_cn.md
+4 -4
View File
@@ -13,7 +13,7 @@ ESP-BOX 开发框架包括系统层、框架层、应用层三个主要层级：
ESP-IDF 是乐鑫官方开源操作系统框架，同时兼容 ESP32、ESP32-S 和 ESP32-C 系列 SoC，完成一次开发即可在多芯片平台部署。它集成了大量的系统级基础组件，用于代码编译和调试的开发工具集，以及详尽的开发指导文档，具体包括：
1. 系统级基础组件，主要包括基于 FreeRTOS 的多任务、多核支持，可外扩 PSRAM 的内存堆分配器；多款 ESP 芯片的 LL、HAL、Driver、VFS 层支持，将不同芯片功能抽象为统一操作接口；多个标准网络协议栈 TCP/IP、HTTP、MQTT、WebSocket 等等；
2. 开发工具集，主要包含用于代码编译的 GCC 交叉工具链、基于 OpenOCD 的 JTAG 调试工具、基于 Segger SystemView 的实时跟踪、Flash 和 eFuse 编程器等；
3. 开发指导文档，详尽的说明了乐鑫各个芯片平台、各个软件版本下的 API 参考，使用指引和注意事项等，开发者可[在线浏览](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/)也可[下载到本地](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/esp-idf-en-v4.4-dev-3540-g4e03a9c-esp32.pdf)查看。
3. 开发指导文档，详尽的说明了乐鑫各个芯片平台、各个软件版本下的 API 参考，使用指引和注意事项等，开发者可[在线浏览](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/)。
### 框架层 Framework Layer
@@ -23,9 +23,9 @@ ESP-IDF 是乐鑫官方开源操作系统框架，同时兼容 ESP32、ESP32-S
ESP-SR 是乐鑫面向智能语音领域开发的语音识别框架，该框架可利用 ESP32-S3 AI 扩展指令进行运算加速，为开发者提供开箱即用的高性能、高可靠性的语音识别功能，它包括：
1. 声学前端（[Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/master/acoustic_algorithm)）算法集，用于提升智能语音设备在远场噪声环境中的交互能力，使开发者获得高质量且稳定的音频数据。它主要包括可以有效滤除扬声器回声的回声消除算法（Acoustic Echo Cancellation，AEC）, 用于强化麦克风阵列方向性的盲源分离算法（Blind Source Separation，BSS），和用于处理环境噪声的噪声抑制算法（Noise Suppression，NS）。ESP-SR 将以上算法封装成简单的 API，开发者无需关心这些算法的具体细节，只需要针对具体的应用场景，对需要使用的算法排列组合，并按照 API 的格式要求输入待处理的语音数据，即可得到声学前端算法的处理结果。
2. 唤醒词识别模型（[WakeNet](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md)），用于在连续的音频流中实时监测出特定的语言片段，将智能设备从待机状态唤醒至语音交互状态。WakeNet 通过计算音频片段的梅尔倒频谱（Mel-Frequency Cepstrum, MFC）作为输入特征，使用针对 ESP32-S3 优化的神经网络算法对特征信号进行分类，在监测到命令词时触发唤醒信号，可在噪声环境下获得不小于 80% 的[有效识别率](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/README.md#performance-test)。ESP-BOX 提供了开箱即用的离线唤醒词 “Hi, ESP” 和 “Hi, 乐鑫”，开发者无需额外的开发工作即可使用，同时乐鑫也支持为开发者[定制独特的唤醒词](https://github.com/espressif/esp-sr/blob/master/wake_word_engine/ESP_Wake_Words_Customization.md)。
3. 命令词识别模型（[MultiNet](https://github.com/espressif/esp-sr/blob/master/speech_command_recognition/README.md)），用于在设备唤醒后，识别用户特定的中英文语音命令，例如 “Turn on the air conditioner”，“关闭电灯” 等。MultiNet 通过基于 CRNN 和 CTC 的卷积循环神经网络，对输入音频片段进行多命令词识别，可同时支持最多 100 个离线命令词。 开发者不需要了解识别模型细节，也无需重新进行模型训练，通过简单配置[中文拼音字符串](https://github.com/espressif/esp-sr/blob/master/speech_command_recognition/README.md#chinese-speech-command-recognition)或[英文音标字符串](https://github.com/espressif/esp-sr/blob/master/speech_command_recognition/README.md#english-speech-command-recognition)以及对应 ID，即可添加或修改命令词，模型在识别到语音命令词后将触发包含 ID 信息的事件。
1. 声学前端（[Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/acoustic_algorithm)）算法集，用于提升智能语音设备在远场噪声环境中的交互能力，使开发者获得高质量且稳定的音频数据。它主要包括可以有效滤除扬声器回声的回声消除算法（Acoustic Echo Cancellation，AEC）, 用于强化麦克风阵列方向性的盲源分离算法（Blind Source Separation，BSS），和用于处理环境噪声的噪声抑制算法（Noise Suppression，NS）。ESP-SR 将以上算法封装成简单的 API，开发者无需关心这些算法的具体细节，只需要针对具体的应用场景，对需要使用的算法排列组合，并按照 API 的格式要求输入待处理的语音数据，即可得到声学前端算法的处理结果。
2. 唤醒词识别模型（[WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)），用于在连续的音频流中实时监测出特定的语言片段，将智能设备从待机状态唤醒至语音交互状态。WakeNet 通过计算音频片段的梅尔倒频谱（Mel-Frequency Cepstrum, MFC）作为输入特征，使用针对 ESP32-S3 优化的神经网络算法对特征信号进行分类，在监测到命令词时触发唤醒信号，可在噪声环境下获得不小于 80% 的[有效识别率](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test)。ESP-BOX 提供了开箱即用的离线唤醒词 “Hi, ESP” 和 “Hi, 乐鑫”，开发者无需额外的开发工作即可使用，同时乐鑫也支持为开发者[定制独特的唤醒词](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/ESP_Wake_Words_Customization.md)。
3. 命令词识别模型（[MultiNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md)），用于在设备唤醒后，识别用户特定的中英文语音命令，例如 “Turn on the air conditioner”，“关闭电灯” 等。MultiNet 通过基于 CRNN 和 CTC 的卷积循环神经网络，对输入音频片段进行多命令词识别，可同时支持最多 200 个离线命令词。 开发者不需要了解识别模型细节，也无需重新进行模型训练，通过简单配置[中文拼音字符串](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands)或[英文音标字符串](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands)以及对应 ID，即可添加或修改命令词，模型在识别到语音命令词后将触发包含 ID 信息的事件。
**ESP-HMI**