docs:update docs for esp32-s3-box-3

This commit is contained in:
marcus_xu
2023-08-22 15:20:00 +08:00
parent 86f3933c11
commit 316004289f
82 changed files with 627 additions and 292 deletions
+37 -24
View File
@@ -1,3 +1,7 @@
<p align="center">
<img src="docs/_static/banner2.jpg" width="auto" height="auto" alt="ESPRESSIF">
</p>
* [中文版本](README_cn.md)
# ESP-BOX AIoT Development Framework
@@ -6,7 +10,7 @@
<a href="https://github.com/espressif/esp-box/blob/master/LICENSE" alt="Build examples">
<img alt="GitHub" src="https://img.shields.io/github/license/espressif/esp-box"></a>
<a href="https://github.com/espressif/esp-box/actions/workflows/build_IDF5.1.yml" alt="Build examples">
<img src="https://github.com/espressif/esp-box/actions/workflows/build_IDF5.1.yml/badge.svg" /></a>
<img src="https://github.com/espressif/esp-box/actions/workflows/build.yml/badge.svg" /></a>
<a href="https://github.com/espressif/esp-box/graphs/contributors" alt="Contributors">
<img src="https://img.shields.io/github/contributors/espressif/esp-box" /></a>
<a href="https://github.com/espressif/esp-box/releases" alt="GitHub all releases">
@@ -19,20 +23,25 @@
**Important Note:**
> We recommend [updating the application firmware](./docs/firmware_update.md) when you first receive the product to have the new features and bugfix. Click [here](./docs/getting_started.md) to access the user guide of the latest release firmware. Click [here](https://github.com/espressif/esp-box/releases) to visit the older firmware.
> Upon receiving the ESP32-S3-BOX and ESP32-S3-BOX-Lite, we advise updating the [built-in factory firmware](./docs/firmware_update.md) to ensure you have the latest bug fixes. In the master branch, we will gradually offer limited support for these two development boards. For further information, kindly refer to the examples readme.
The ESP-BOX is a new generation AIoT development platform released by Espressif Systems. The ESP32-S3-BOX and ESP32-S3-BOX-Lite are AIoT development boards that are based on Espressifs ESP32-S3 Wi-Fi + Bluetooth 5 (LE) SoC. They provide a platform for developing the control of home appliances using Voice Assistance + touch and screen controller, sensor, infrared controller, and intelligent Wi-Fi gateway. Development boards come with pre-built firmware that supports offline voice interaction, with the SDKs and examples provided by Espressif, you will be able to develop a wide variety of AIoT applications based on the BOX series products such as online and offline voice assistants, voice-enabled devices, human-computer interaction devices, control panels, multi-protocol gateways easily.
The ESP-BOX is an advanced AIoT, Edge AI, and IIoT applications development platform released by Espressif Systems. The ESP32-S3-BOX, ESP32-S3-BOX-Lite, and ESP32-S3-BOX-3 are a series of development boards designed for this platform, they are built on Espressifs powerful ESP32-S3 Wi-Fi + Bluetooth 5 (LE) SoC and feature compact and beautiful enclosures. Leveraging the multi-functional accessories and reliable examples from the repos, these development boards cater to both prototyping new projects and building complex IoT systems, providing a perfect balance of form and function.
ESP-BOX provides a platform for developing applications in various domains, including Offline Voice Assistants, Online AI Chatbots (developed using OpenAI and other large language model platforms), Matter Device/Controller, Robot Controller, USB Device/Host, Wireless Sensor Application and a wide range of Human Machine Interaction (HMI) applications. Experience the possibilities—add one to your toolkit and explore the limitless potential!
| Product Name | Product Figure | Marketing Status |
| :-----: | :---------------------: |:---------------------: |
| [ESP32-S3-BOX](docs/hardware_overview/esp32_s3_box/hardware_overview_for_box.md) | <img src="docs/_static/esp32_s3_box.png" width="200px" /> |End of Life <br> [User Guide](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started.md) |
| [ESP32-S3-BOX-Lite](docs/hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite.md) | <img src="docs/_static/esp32_s3_box_lite.png" width="200px" /> |[Active](https://www.aliexpress.com/item/1005004441701974.html?spm=a2g0o.store_pc_groupList.8148356.15.378130cf2Qwt87&pdp_npi=3%40dis%21HKD%21HKD273.50%21HKD273.50%21%21%21%21%21%40210318cb16926755466344763e3593%2112000029194032303%21sh%21HK%213721963538) <br> [User Guide](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started.md)|
| [ESP32-S3-BOX-3](docs/hardware_overview/esp32_s3_box_3/hardware_overview_for_box_3.md) | <img src="docs/_static/esp32_s3_box_3.png" width="200px" /> |[Active](https://www.aliexpress.com/item/1005005920207976.html) <br> [User Guide](./docs/getting_started.md)|
| Product Name | Product Figure |
| :-----: | :---------------------: |
| [ESP32-S3-BOX](docs/hardware_overview/esp32_s3_box/hardware_overview_for_box.md) | <img src="docs/_static/esp32_s3_box.png" width="200px" /> |
| [ESP32-S3-BOX-Lite](docs/hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite.md) | <img src="docs/_static/esp32_s3_box_lite.png" width="200px" /> |
## Versions
| ESP-BOX | Dependent ESP-IDF | Branch Note | Support State |
| :--------------: | :---------------: | :----------------------------------------------------: | -------------- |
| master | >= release/v5.1<br/>commit id: 22cfbf30c3 | Latest developing firmware <br/>esp-sr components version: v1.3.4 | Compatible with ESP32-S3-BOX and ESP32-S3-BOX-Lite <br/>Use menuconfig to select board |
| master | >= release/v5.1<br/>commit id: 22cfbf30c3 | Latest developing firmware <br/>esp-sr components version: v1.4.1 | Partial support for ESP32-S3-BOX, ESP32-S3-BOX-Lite, and ESP32-S3-BOX-3 <br> Refer to the Examples **README** for more details. |
| Tag 0.5.0 | release/v5.1<br>commit id: 22cfbf30c3 | esp-sr components version: v1.3.4 | Compatible with ESP32-S3-BOX and ESP32-S3-BOX-Lite <br/>Use menuconfig to select board |
| Tag 0.3.0 | release/v4.4<br>commit id: 2bdea81b2a | esp-sr version: dev/v2.0<br>commit id: c873a35 | Compatible with ESP32-S3-BOX and ESP32-S3-BOX-Lite |
| Tag 0.2.1 | release/v4.4 with [patch](https://github.com/espressif/esp-box/tree/v0.2.1/idf_patch) | esp-sr version: close v1.0<br/>commit id: 3ce34fe | Support ESP32-S3-BOX only |
@@ -40,30 +49,34 @@ The ESP-BOX is a new generation AIoT development platform released by Espressif
## Features
* Supports far-field voice interaction with 2 mics
* Offline voice wake-up with high wake-up rate
* Offline speech commands recognition in Chinese and English languages with high recognition rate
* Reconfigurable 200+ speech commands in Chinese and English languages
* Continuous Identification and Wakeup Interrupt
* Flexible and reusable GUI framework
* End-to-end AIoT dvelopment framwork ESP-RainMaker
* Pmod™ compatible headers support expand peripheral modules
* An out-of-the-box entry-level edge AI + HMI application development board.
* FreeRTOS-based four-in-one voice interaction panel: offline voice recognition, network communication, screen display, peripheral control.
* Supports far-field voice interaction with 2 mics.
* Offline voice wake-up and commands recognition with high wake-up rate.
* Allows continuous recognition, wake-up interrupt, and over 200 customizable command words.
* Flexible and versatile home automation solutions: Matter, Home Assistant, ESP-RainMaker
* Visual drag-and-drop GUI development: LVGL SquareLine Studio, Embedded Wizard, etc.
* Rich development framework: ESP-IDF, Arduino, PlatformIO, Circuit Python, and more.
## Open-Source Content
* [Schematic and PCB Source File](./hardware)
* [Shell 3D Print Source File](./hardware)
* [Firmware Source Code](./examples/factory_demo)
* [Examples Source Code](./examples)
* [Built-in Firmware Source Code](./examples/factory_demo)
* [Other Examples](./examples)
> Based on [Apache 2.0](https://github.com/espressif/esp-box/blob/master/LICENSE) open source license, you can modify, develop, or distribute the contents freely.
## User Guide
## Built-in Firmware User Guide
* For getting start of BOX serial development boards, please refer to the [Getting Started](./docs/getting_started.md).
* For experiencing voice assistant, please refer to the [Voice Assistance Control and Customization](./docs/getting_started.md#offline-voice-assistant-with-your-device).
* For experiencing a more humanized interactive experience, please refer to the [Continuous Speech Recognition](./docs/getting_started.md#continuous-recognition).
* It is recommended to start by reading the product [Hardware Overview](./docs/hardware_overview/esp32_s3_box_3/hardware_overview_for_box_3.md).
* Next you can explore the [Getting Started](./docs/getting_started.md).
* For exploring interesting sensor demos, please refer to [Sensor Applications](./docs/getting_started.md#sensor-monitor).
* For experiencing voice assistant, please refer to the [Voice Assistance Control](./docs/getting_started.md#offline-voice-assistant).
* For experiencing a more humanized interactive experience, please refer to the [Continuous Speech Recognition](./docs/getting_started.md#continuous-speech-recognition).
* For defining the speech commands by mobile APP, please refer to the [ESP BOX APP User Guide](./docs/getting_started.md#voice-command-customization).
* For switching the voice model language, please refer to [Switch Voice Model Language](./docs/getting_started.md#switch-voice-model-language).
* For product exploded-view and disassembly tutorial, please refer to [Disassembly Tutorial](docs/disassembly_tutorial.md).
* For more details of hardware information, please refer to the [Hardware Overview](./docs/hardware_overview).
* For latest version firmware, please refer to the [Firmware Update](./docs/firmware_update.md).
@@ -74,8 +87,8 @@ ESP-BOX integrates components commonly used in AIoT development that you can lev
### Quick Start
* **Step 1**. ESP-IDF (Espressif IoT Development Framework) is required to build the application. If this is your first time using the ESP-IDF, please refer to [ESP-IDF development guide](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/index.html) to understand the open-source drivers and components provided by Espressif.
* **Step 2**. Next, setting up development environment. Please refer to [ESP-IDF(release/v5.1) Installation Step by Step](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/get-started/index.html).
* **Step 1**. ESP-IDF (Espressif IoT Development Framework) is required to build the application. If this is your first time using the ESP-IDF, please refer to [ESP-IDF development guide](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/index.html) to understand the open-source drivers and components provided by Espressif.
* **Step 2**. Next, setting up development environment. Please refer to [ESP-IDF(release/v5.1) Installation Step by Step](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/get-started/index.html#ide).
* **Step 3**. Run command `git clone --recursive https://github.com/espressif/esp-box.git` to download the code of this project and then switch the ESP-IDF version to [specified version](#versions);
* **Step 4**. Take a small step, you can [build and flash a simple example](./examples/image_display).
* **Step 5**. Moving further, please read [ESP-BOX technical architecture](./docs/technical_architecture.md) to understand the technical details.
+106 -93
View File
@@ -1,93 +1,106 @@
* [English Version](./README.md)
* [Gitee 镜像仓库](https://gitee.com/EspressifSystems/esp-box)
# ESP-BOX AIoT 开发框架
<p align="left">
<a href="https://github.com/espressif/esp-box/blob/master/LICENSE" alt="Build examples">
<img alt="GitHub" src="https://img.shields.io/github/license/espressif/esp-box"></a>
<a href="https://github.com/espressif/esp-box/actions/workflows/build_IDF5.1.yml" alt="Build examples">
<img src="https://github.com/espressif/esp-box/actions/workflows/build_IDF5.1.yml/badge.svg" /></a>
<a href="https://github.com/espressif/esp-box/graphs/contributors" alt="Contributors">
<img src="https://img.shields.io/github/contributors/espressif/esp-box" /></a>
<a href="https://github.com/espressif/esp-box/releases" alt="GitHub all releases">
<img src="https://img.shields.io/github/downloads/espressif/esp-box/total" /></a>
</p>
<a href="https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml">
<img alt="Try it with ESP Launchpad" src="https://espressif.github.io/esp-launchpad/assets/try_with_launchpad.png" width="200" height="56">
</a>
**重要提醒:**
> 我们建议您在第一次收到产品时[更新应用固件](./docs/firmware_update_cn.md)以获得最新功能和 bug 的修复,点此[链接](./docs/getting_started_cn.md)获取应用固件的使用说明。点击[此处](https://github.com/espressif/esp-box/releases)查看历史固件。
ESP-BOX 是乐鑫信息科技发布的新一代 AIoT 应用开发平台。ESP32-S3-BOX 和 ESP32-S3-BOX-Lite 是目前对应的 AIoT 应用开发板,搭载支持 AI 加速的 ESP32-S3 Wi-Fi + Bluetooth 5 (LE) SoC。他们为用户提供了一个基于语音助手 + 触摸屏控制、传感器、红外控制器和智能 Wi-Fi 网关等功能,开发和控制智能家居设备的平台。开发板出厂支持离线语音交互功能,用户通过乐鑫丰富的 SDK 和解决方案,能够轻松构建在线和离线语音助手、智能语音设备、HMI 人机交互设备、控制面板、多协议网关等多样的应用。
| 开发板名称 | 产品图 |
| :-----: | :---------------------: |
| [ESP32-S3-BOX](docs/hardware_overview/esp32_s3_box/hardware_overview_for_box_cn.md) | <img src="docs/_static/esp32_s3_box.png" width="200px" /> |
| [ESP32-S3-BOX-Lite](docs/hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite_cn.md) | <img src="docs/_static/esp32_s3_box_lite.png" width="200px" /> |
## 版本信息
| ESP-BOX | 依赖的 ESP-IDF | 分支信息 | 支持状态 |
| :-------: | :----------------------------------------------------------: | :----------------------------------------------------------: | ------------------------------------------------------- |
| master | >= release/v5.1<br/>commit id: 22cfbf30c3 | Latest developing firmware <br/>esp-sr components version: v1.3.4 | 兼容 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite<br/>使用 menuconfig 选择开发板 |
| Tag 0.5.0 | release/v5.1<br/>commit id: 22cfbf30c3 | esp-sr components version: v1.3.4 | 兼容 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite<br/>使用 menuconfig 选择开发板 |
| Tag 0.3.0 | release/v4.4<br/>commit id: 22cfbf30c3 | esp-sr version: dev/v2.0<br/>commit id: c873a35 | 兼容 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite |
| Tag 0.2.1 | release/v4.4 with [patch](https://github.com/espressif/esp-box/tree/v0.2.1/idf_patch) | esp-sr version: close v1.0<br/>commit id: 3ce34fe | 仅支持 ESP32-S3-BOX |
| Tag 0.1.1 | Release/v4.4 with [patch](https://github.com/espressif/esp-box/tree/v0.1.1/idf_patch) | esp-sr version: v0.9.6<br/>commit id: 3ce34fe | 仅支持 ESP32-S3-BOX |
## 支持特性
* 双麦克风支持远场语音交互
* 高唤醒率的离线语音唤醒
* 高识别率的离线中英文命令词识别
* 可动态配置 200+ 中英文命令词
* 连续识别和唤醒打断
* 灵活可复用的 GUI 框架
* 端到端一站式接入云平台
* Pmod™ 兼容接口支持多种外设扩展
## 开源内容
* [原理图与 PCB 源文件](./hardware)
* [外壳 3D 打印源文件](./hardware)
* [出厂固件源代码](./examples/factory_demo)
* [示例程序源代码](./examples)
> 基于 [Apache 2.0](https://github.com/espressif/esp-box/blob/master/LICENSE) 开源协议,你可以免费且自由的修改、创作和分享。
## 使用指引
* 首次拿到 BOX 系列开发板,您可以查看 [首次使用操作指引](./docs/getting_started_cn.md#给设备供电)
* 想要体验离线语音助手功能,您可以查看 [离线语音识别](./docs/getting_started_cn.md#体验离线语音识别)
* 体验更人性化的语音交互,可查看 [连续语音识别](./docs/getting_started_cn.md#连续语音识别)
* 想要通过手机 APP 设置独特的语音命令词,请查看 [ESP BOX APP 操作指引](./docs/getting_started_cn.md#语音命令词自定义)
* 想要了解产品爆炸图和拆机说明,请查看 [拆机教程](docs/disassembly_tutorial.md)
* 想要了解开发板硬件和接口细节,请查看 [硬件总览](./docs/hardware_overview)
* 想要使用最新版本固件,请查看 [固件更新说明](./docs/firmware_update_cn.md)。
## 开发指引
ESP-BOX 调用乐鑫包管理器 ESP Registry 中常用的组件,您可以利用这些组件构建自己的应用。点击了解更多[开发指南](./docs/development_guide_cn.md)(持续更新中)。
### 快速开始
* **Step 1**. 如果您首次接触 ESP-IDF 开发,建议先浏览 [ESP-IDF 编程指南](https://docs.espressif.com/projects/esp-idf/zh_CN/release-v5.1/esp32s3/index.html),了解乐鑫已经提供的开源驱动和组件
* **Step 2**. 接下来您可以详细阅读 [ESP-IDFrelease/v5.1 环境搭建指引](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/get-started/index.html),一步一步完成开发环境搭建;
* **Step 3**. 使用指令 `git clone --recursive https://github.com/espressif/esp-box.git` 下载本项目的代码,然后需要将 esp-idf 版本切换到[指定的版本](#版本信息)
* **Step 4**. 小试牛刀,您可以尝试 [构建并烧录一个新的示例程序](./examples/image_display)
* **Step 5**. 更进一步,您可以阅读 [ESP-BOX 技术架构说明](./docs/technical_architecture_cn.md),了解更多技术细节
* **Step 6**. 深入开发,您可以阅读 [ESP 语音识别应用开发指南](https://github.com/espressif/esp-sr)[ESP RainMaker 编程指南](https://docs.espressif.com/projects/esp-rainmaker/en/latest/)[LVGL UI 界面开发快速总览](https://docs.lvgl.io/8.3/)
* **Step 7**. 扩展功能,您可以阅读 [开发板 Pmod™ 兼容接口介绍](./docs/hardware_overview) 和 [ESP-IDF API 参考手册](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/api-reference/index.html),为更多扩展设备开发驱动。
## 联系我们
* 如果有任何开发和使用问题,您可添加 [GitHub Issues](https://github.com/espressif/esp-box/issues) 或到 [esp32.com](https://esp32.com/) 论坛寻求帮助;
* 如果你对 ESP-BOX 项目感兴趣,欢迎到论坛 [ESP-BOX 版块](https://www.esp32.com/viewforum.php?f=43) 交流想法。
## 贡献示例
我们欢迎任何软件或硬件相关的开源贡献,如果您有任何要与我们分享的示例,请参考 [贡献指南](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/contribute/index.html) 后提交 [Pull Requests](https://github.com/espressif/esp-box/pulls) 贡献代码。
<p align="center">
<img src="docs/_static/banner2.jpg" width="auto" height="auto" alt="ESPRESSIF">
</p>
* [English Version](./README.md)
* [Gitee 镜像仓库](https://gitee.com/EspressifSystems/esp-box)
# ESP-BOX AIoT 开发框架
<p align="left">
<a href="https://github.com/espressif/esp-box/blob/master/LICENSE" alt="Build examples">
<img alt="GitHub" src="https://img.shields.io/github/license/espressif/esp-box"></a>
<a href="https://github.com/espressif/esp-box/actions/workflows/build_IDF5.1.yml" alt="Build examples">
<img src="https://github.com/espressif/esp-box/actions/workflows/build.yml/badge.svg" /></a>
<a href="https://github.com/espressif/esp-box/graphs/contributors" alt="Contributors">
<img src="https://img.shields.io/github/contributors/espressif/esp-box" /></a>
<a href="https://github.com/espressif/esp-box/releases" alt="GitHub all releases">
<img src="https://img.shields.io/github/downloads/espressif/esp-box/total" /></a>
</p>
<a href="https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml">
<img alt="Try it with ESP Launchpad" src="https://espressif.github.io/esp-launchpad/assets/try_with_launchpad.png" width="200" height="56">
</a>
**重要提醒:**
> 对于 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite,我们建议您更新内置的[出厂应用固件](./docs/firmware_update_cn.md)以确保拥有最新的 bug 修复。在 Master 分支,我们将逐步对这两款开发板提供有限的支持。有关更多信息,请阅读各例程的 README。
ESP-BOX 是乐鑫科技推出的面向 AIoT、边缘 AI 和 IIoT 应用的开发平台。ESP32-S3-BOX、ESP32-S3-BOX-Lite 和 ESP32-S3-BOX-3 是为该平台设计的系列开发板,它们基于乐鑫强大的 ESP32-S3 Wi-Fi + Bluetooth 5LE SoC 构建,并拥有紧凑美观的外壳结构。借助多功能配件和 ESP-BOX 项目内的可靠例程,这些开发板既适用于构建新项目原型,也适用于打造复杂的物联网系统,提供了形式上和功能上的完美平衡。
ESP-BOX 为各个领域的应用提供了开发资源,包括离线语音助手、在线 AI 聊天机器人(使用 OpenAI 和其他大型语言模型平台开发)、Matter 设备/控制器、机器人控制器、USB 主从设备、无线传感器应用以及广泛的人机交互(HMI)应用。将其加入工具箱,尽情体验各种可能性,发掘无限潜力!
| 产品名称 | 产品图 | 市场状态 |
| :-----: | :---------------------: |:---------------------: |
| [ESP32-S3-BOX](docs/hardware_overview/esp32_s3_box/hardware_overview_for_box_cn.md) | <img src="docs/_static/esp32_s3_box.png" width="200px" /> |寿命终止 <br> [用户指南](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started_cn.md) |
| [ESP32-S3-BOX-Lite](docs/hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite_cn.md) | <img src="docs/_static/esp32_s3_box_lite.png" width="200px" /> |[在售](https://item.taobao.com/item.htm?spm=a312a.7700824.w4002-8715811646.9.4048605fNqggSF&id=658634202331) <br> [用户指南](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started_cn.md)|
| [ESP32-S3-BOX-3](docs/hardware_overview/esp32_s3_box_3/hardware_overview_for_box_3_cn.md) | <img src="docs/_static/esp32_s3_box_3.png" width="200px" /> |[在售](https://item.taobao.com/item.htm?ft=t&id=732842971319) <br> [用户指南](./docs/getting_started_cn.md)|
## 版本信息
| ESP-BOX | 依赖的 ESP-IDF | 分支信息 | 支持状态 |
| :-------: | :----------------------------------------------------------: | :----------------------------------------------------------: | ------------------------------------------------------- |
| master | >= release/v5.1<br/>commit id: 22cfbf30c3 | Latest developing firmware <br/>esp-sr components version: v1.4.1 | 部分支持 ESP32-S3-BOXESP32-S3-BOX-Lite 和 ESP32-S3-BOX-3<br/>详见 Examples **README** |
| Tag 0.5.0 | release/v5.1<br/>commit id: 22cfbf30c3 | esp-sr components version: v1.3.4 | 兼容 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite<br/>使用 menuconfig 选择开发板 |
| Tag 0.3.0 | release/v4.4<br/>commit id: 22cfbf30c3 | esp-sr version: dev/v2.0<br/>commit id: c873a35 | 兼容 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite |
| Tag 0.2.1 | release/v4.4 with [patch](https://github.com/espressif/esp-box/tree/v0.2.1/idf_patch) | esp-sr version: close v1.0<br/>commit id: 3ce34fe | 仅支持 ESP32-S3-BOX |
| Tag 0.1.1 | Release/v4.4 with [patch](https://github.com/espressif/esp-box/tree/v0.1.1/idf_patch) | esp-sr version: v0.9.6<br/>commit id: 3ce34fe | 仅支持 ESP32-S3-BOX |
## 支持特性
* 即开即用的入门级边缘 AI + HMI 应用开发板
* 基于 FreeRTOS 的四合一语音交互面板:离线语音识别、网络通信、屏幕显示、外设控制
* 双麦克风支持远场语音交互
* 支持高唤醒率的离线语音唤醒和命令词识别
* 允许连续命令词识别、唤醒打断以及自定义配置 200+ 中英文命令词
* 灵活多样的家庭自动化解决方案:Matter、Home Assistant、ESP-RainMaker
* 可视化拖放式 GUI 开发:LVGL SquareLine Studio、Embedded Wizard 等
* 丰富的开发框架:ESP-IDF、Arduino、PlatformIO、Circuit Python 等
## 开源内容
* [原理图与 PCB 源文件](./hardware)
* [外壳 3D 打印源文件](./hardware)
* [出厂固件源代码](./examples/factory_demo)
* [示例程序源代码](./examples)
> 基于 [Apache 2.0](https://github.com/espressif/esp-box/blob/master/LICENSE) 开源协议,你可以免费且自由的修改、创作和分享。
## 使用指引
* 首次拿到开发板,建议先阅读产品[硬件概览](./docs/hardware_overview)
* 接着可以查看[首次使用操作指引](./docs/getting_started_cn.md#开始使用)
* 体验有意思的[传感器应用](./docs/getting_started_cn.md#传感器监测)
* 体验离线语音助手功能,您可以查看[离线语音识别](./docs/getting_started_cn.md#体验离线语音识别)
* 体验更人性化的语音交互,可查看[连续语音识别](./docs/getting_started_cn.md#连续语音识别)
* 通过手机 APP 设置独特的语音命令词,请查看 [ESP BOX APP 操作指引](./docs/getting_started_cn.md#语音命令词自定义)
* 如需切换语音模型语言,请查看[中英文语音模型切换](./docs/getting_started_cn.md#中英文语音模型切换)
* 了解产品爆炸图和拆机说明,请查看[拆机教程](./docs/disassembly_tutorial.md)
* 更新最新版本固件,请查看[固件更新说明](./docs/firmware_update_cn.md)
## 开发指引
ESP-BOX 调用乐鑫包管理器 ESP Registry 中常用的组件,您可以利用这些组件构建自己的应用。点击了解更多[开发指南](./docs/development_guide_cn.md)(持续更新中)。
### 快速开始
* **Step 1**. 如果您首次接触 ESP-IDF 开发,建议先浏览 [ESP-IDF 编程指南](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32s3/index.html),了解乐鑫已经提供的开源驱动和组件;
* **Step 2**. 接下来您可以详细阅读 [ESP-IDFrelease/v5.1) 环境搭建指引](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/get-started/index.html#ide),一步一步完成开发环境搭建;
* **Step 3**. 使用指令 `git clone --recursive https://github.com/espressif/esp-box.git` 下载本项目的代码,然后需要将 esp-idf 版本切换到[指定的版本](#版本信息)
* **Step 4**. 小试牛刀,您可以尝试 [构建并烧录一个新的示例程序](./examples/image_display)
* **Step 5**. 更进一步,您可以阅读 [ESP-BOX 技术架构说明](./docs/technical_architecture_cn.md),了解更多技术细节;
* **Step 6**. 深入开发,您可以阅读 [ESP 语音识别应用开发指南](https://github.com/espressif/esp-sr)[ESP RainMaker 编程指南](https://docs.espressif.com/projects/esp-rainmaker/en/latest/)[LVGL UI 界面开发快速总览](https://docs.lvgl.io/8.3/)
* **Step 7**. 扩展功能,您可以阅读 [开发板 Pmod™ 兼容接口介绍](./docs/hardware_overview) 和 [ESP-IDF API 参考手册](https://docs.espressif.com/projects/esp-idf/en/release-v5.1/esp32s3/api-reference/index.html),为更多扩展设备开发驱动。
## 联系我们
* 如果有任何开发和使用问题,您可添加 [GitHub Issues](https://github.com/espressif/esp-box/issues) 或到 [esp32.com](https://esp32.com/) 论坛寻求帮助;
* 如果你对 ESP-BOX 项目感兴趣,欢迎到论坛 [ESP-BOX 版块](https://www.esp32.com/viewforum.php?f=43)交流想法。
## 贡献示例
我们欢迎任何软件或硬件相关的开源贡献,如果您有任何要与我们分享的示例,请参考[贡献指南](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/contribute/index.html)后提交 [Pull Requests](https://github.com/espressif/esp-box/pulls) 贡献代码。

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 210 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 242 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 376 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 267 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 295 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 388 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 184 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 269 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 354 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 110 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

+1 -1
View File
@@ -111,7 +111,7 @@ Includes the following components:
**bsp_board.c** is the system call file used to be compatible with different hardware platforms. Currently, the box project supports two hardware platforms: [BOARD_S3_BOX](./hardware_overview/esp32_s3_box/hardware_overview_for_box.md) and [BOARD_S3_BOX_LITE](./hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite.md). After the system initialization, based on the different device addresses detected by IIC, it determines the current hardware platform and calls different hardware initialization interfaces accordingly.
**esp32_s3_box.c** and **esp32_s3_box_lite.c** are the specific hardware pin configurations and initialization implementations for `BOARD_S3_BOX` and `BOARD_S3_BOX_LITE`, respectively. Users need to manually select the hardware platform they are running on in menuconfig to invoke the corresponding hardware initialization interface.
**esp32_s3_box.c** and **esp32_s3_box_lite.c** are the specific hardware pin configurations and initialization implementations for `BOARD_S3_BOX` and `BOARD_S3_BOX_LITE`, respectively.
- codec
+3 -1
View File
@@ -112,7 +112,9 @@ idf.py -p PORT flash monitor
### bsp 组件
- boards
**esp32_s3_box.c** 和 **esp32_s3_box_lite.c** 分别是 `BOARD_S3_BOX``BOARD_S3_BOX_LITE` 的具体硬件管脚配置和初始化实现。用户需要在 menuconfig 中手动选择当前运行在哪个硬件平台,以分别调用不同的硬件初始化接口。
**bsp_board.c** 是系统调用文件,用以兼容不同硬件平台,目前 box 工程支持 [BOARD_S3_BOX](hardware_overview/esp32_s3_box/hardware_overview_for_box_cn.md)[BOARD_S3_BOX_LITE](hardware_overview/esp32_s3_box_lite/hardware_overview_for_lite_cn.md) 两个硬件平台。系统初始化后,根据 iic detect 到的不同器件地址判断当前运行在哪个硬件平台,以分别调用不同的硬件初始化接口。
**esp32_s3_box.c** 和 **esp32_s3_box_lite.c** 分别是 `BOARD_S3_BOX` 和 `BOARD_S3_BOX_LITE` 的具体硬件管脚配置和初始化实现。
- codec
+2 -2
View File
@@ -3,7 +3,7 @@
# Updating Firmware
**Warning:**
> The firmware released in the latest Tag branch (Version v0.5.0) is only applicable to ESP32-S3-BOX and ESP32-S3-BOX-Lite. We suggest that users update the firmware via Espressif's online firmware flashing tool **[ESP Launchpad](https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml)**.
> The firmware released in the latest Tag branch ([Version v0.5.0](https://github.com/espressif/esp-box/releases)) is only applicable to ESP32-S3-BOX and ESP32-S3-BOX-Lite. For users who have purchased ESP32-S3-BOX-3, we will release firmware updates during the project's progression. We suggest that users update the firmware via Espressif's online firmware flashing tool **[ESP Launchpad](https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml)**.
The BOX series development boards support updating firmware through USB. To update firmware, you only need:
@@ -38,7 +38,7 @@ Please follow the link below corresponding to your OS to update the firmware.
ESP Launchpad is a web based tool, available for flashing firmware application to the ESP32 device connected via USB serial port. You can just click [here](https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml) to start.
1. Connect your development board to the computer through a USB Type-C cable.
2. `Connect` the device device follow indicated steps in the figure below.
2. `Connect` the device follow indicated steps in the figure below.
3. Choose a bin from `Select Application` and click `Flash` to update.
<div align="center">
+1 -1
View File
@@ -2,7 +2,7 @@
# 更新固件
**警告:**
> 当前 Tag 分支 (最新 Tag v0.5.0)发布的固件仅适用于 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite。我们推荐用户通过乐鑫在线固件烧录工具 **[ESP Launchpad](https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml)** 来更新固件。
> 当前 Tag 分支 (最新 [Tag v0.5.0](https://github.com/espressif/esp-box/releases))发布的固件仅适用于 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite,对于购买了 ESP32-S3-BOX-3 用户,在项目进展期间我们会发布可更新的固件。我们推荐用户通过乐鑫在线固件烧录工具 **[ESP Launchpad](https://espressif.github.io/esp-launchpad/?flashConfigURL=https://raw.githubusercontent.com/espressif/esp-box/master/launch.toml)** 来更新固件。
BOX 系列开发板支持使用 USB 接口更新固件,您仅需要准备:
1. 一根 USB Type-C 数据线
+131 -70
View File
@@ -1,63 +1,76 @@
<div align="center">
<img src="./_static/_get_started_static/started_banner.png" width="700px">
</div>
* [中文版本](./getting_started_cn.md)
The guide is applicable to ESP32-S3-BOX Kits and ESP32-S3-BOX-Lite Kits with the latest version of firmware flashed. They are collectively referred to as BOX series of development boards in this guide.
The guide is applicable to ESP32-S3-BOX-3 only. For the user guide to the ESP32-S3-BOX and ESP32-S3-BOX-Lite built-in firmware, please refer to the [latest release Tag branch](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started.md).
# Overview
In a world where innovation combines with artistry, where sharing and collaboration are paramount, we introduce the ESP32-S3-BOX-3, a fully open-source AIoT development kit based on the powerful ESP32-S3 SoC. This next-generation tool brings a positive change to traditional development boards, it features a compact and beautiful enclosure also provides rich assemblies, empowering developers to easily customize and expand its functionality with Espressif trusted software solutions such as ESP-BOX, ESP-SR, ESP-Rainmaker and ESP-Matter. Whether you're prototyping a new project or building a complex IoT system, the ESP32-S3-BOX-3 strikes the perfect balance of form and function, propelling the future of development board ecology.
The ESP32-S3-BOX-3 comes with pre-built firmware supporting offline voice wake-up and speech recognition in both Chinese and English. The ESP-BOX mobile APP facilitates reconfigurable AI voice interaction, enabling you to create personalized commands for controlling smart devices. Furthermore, the firmware showcases various sensor demos and enables users to perform IR learning, transforming the box into a controller for in-house air conditioners.
This guide provides a comprehensive overview of the built-in firmware's capabilities to get you started on your journey. As you read through, you'll gain valuable experience on developing your applications independently. Let's embark on this exciting adventure together!
# Target Group
The target group for this product includes individuals and organizations engaged in making, prototyping, and development activities. It is designed to cater to a wide range of users, such as hobbyists, makers, engineers, educators, and professionals seeking versatile and expandable solutions for their projects.
# Target Application
* How to call these things, use one trem: Smart Home
* Smart Agriculture
* Smart City
* Energy Management
* IoT Prototyping and Education
* Industrial Internet of Things
* Smart Retail
<div align="center">
<img src="./_static/_get_started_static/application.png" width="700px">
</div>
# Getting Started
The BOX series of development boards integrated with ESP32-S3 SoCs provide a platform for users to develop the control system of smart home appliances, using the voice assistance + touch screen controller, sensor, infrared remote controller, and intelligent Wi-Fi gateway. The BOX series of development boards come with the pre-built firmware that supports voice wake-up and offline speech recognition both in Chinese and English. The ESP-BOX SDK features reconfigurable AI voice interaction, allowing you to customize commands to control home appliances. This guide briefly introduces what you can do with the latest version of firmware to help you get started. After you read through the guide, you may start developing an application by yourself. Now, lets begin!
**The BOX Kit contains:**
| ESP32-S3-BOX | ESP32-S3-BOX-Lite |
| :-----: | :---------------------: |
| A main unit that can work on its own | A main unit that can work on its own|
| An RGB LED module and Dupont wires for testing | An RGB LED module and Dupont wires for testing|
| Dock, an accessory that serves as a stand for the main unit | N/A|
**Required Hardware:**
Please find yourself a USB-C cable.
## Connect the RGB LED Module to Your Device
Please refer to the pin definition below, and connect the RGB LED module to the BOX using DuPont wires. The module has four male pins, R, G, B, and GND. Please connect them to the female ports G39, G40, G41, and GND on PMOD 1.
## Contents of the Kit
<div align="center">
<img src="./_static/_get_started_static/hardware_pmod.png">
<img src="./_static/_get_started_static/ESP32_S3_BOX_3_KitFigure.png" width="700px">
</div>
- ESP32-S3-BOX-3, a main unit that can work on its own
- ESP32-S3-BOX-3-DOCK, a functional accessory serving as a stand for the main box
- ESP32-S3-BOX-3-SENSOR, a functional accessory showcasing sensor applications
- ESP32-S3-BOX-3-BRACKET, an adapter accessory for mounting the main box to other devices
- ESP32-S3-BOX-3-BREAD, an adapter accessory facilitating easy connection of the main box to a standard breadboard
- A USB-C Power Cable
- An RGB LED module and Dupont wires for testing
## Power on Your Device
1. Power on your device using the USB-C cable.
<div align="center">
<img src="./_static/_get_started_static/usb_power.png" width="800px">
<img src="./_static/_get_started_static/usb_power_box.png" width="800px">
</div>
2. Once the device is powered on, the screen will play the Espressif logo boot animation.
<div align="center">
<img src="./_static/_get_started_static/boot_animation.png" width="700px">
<img src="./_static/_get_started_static/boot_animation.png" width="500px">
</div>
## Let's Play Around!
1. The first two pages of the quick guide introduce what the buttons do on your BOX series of development boards. Tap `Next` to go to the next page.
1. The initial two pages of the quick guide provide an overview of the functions of the buttons on your BOX. Press the `Next` button to proceed to the subsequent page.
<table border="0" align="center">
<tr>
<td><b>ESP32-S3-BOX </b></td>
<td><b>ESP32-S3-BOX-Lite </b></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/0.png" width="300px"/></td>
<td><img src="./_static/_get_started_static/1.png" width="300px"/></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/2.png" width="280px"/></td>
<td><img src="./_static/_get_started_static/2.png" width="280px"/></td>
<td><img src="./_static/_get_started_static/0.png"></td>
<td><img src="./_static/_get_started_static/2.png"></td>
</tr>
</table>
2. The last two pages of the quick guide introduces how to use AI voice control. Tap `OK Lets Go` to enter the menu.
2. The subsequent pages of the quick guide introduce how to use AI voice control. Tap `OK Lets Go` to access the menu.
<table border="0" align="center">
<tr>
@@ -66,7 +79,7 @@ Please refer to the pin definition below, and connect the RGB LED module to the
</tr>
</table>
3. There are five options in the menu: `Device Control`, `Network`, `Media Player`, `Help`, and `About Us`. You can navigate to different options by swiping left and right. For example, enter `Device Control` screen, tap `Light` to turn on or off the LED light on the module. Then you may go back to the menu, and enter `Media Player` screen to play music or adjust the volume.
3. The menu consists of six options: `Sensor Monitor`, `Device Control`, `Network`, `Media Player`, `Help`, and `About Us`. You can move between these options by swiping left or right. For instance, you can access the `Device Control` screen, tap on `Light` to toggle the light on or off. Afterward, you can return to the menu, enter the `Media Player` screen, and either play music or adjust the system volume.
<table border="0" align="center">
<tr>
@@ -79,65 +92,113 @@ Please refer to the pin definition below, and connect the RGB LED module to the
</tr>
</table>
**Only ESP32-S3-BOX supports the following features:**
4. Press the mute button on the top of the device to disable voice wake-up and speech recognition. Press again to enable them.
4. Press the mute button on the top of the device to disable voice wake-up and speech recognition. Press it again to enable them.
<div align="center">
<img src="_static/_get_started_static/hardware_mute_button.jpg" width="350px">
<img src="./_static/_get_started_static/hardware_mute.png" width="350px">
</div>
5. Tap the red circle located below the screen to go back to the previous page. This red circle serves as a customizable touch region, allowing users to customize extra touch capabilities according to their preferences.
<div align="center">
<img src="./_static/_get_started_static/hardware_home.png" width="350px">
<img src="./_static/_get_started_static/hardware_home.png" width="250px">
</div>
## Sensor Monitor
This feature needs ESP32-S3-BOX-3 is mounted to the ESP32-S3-BOX-3-SENSOR dock. ESP32-S3-BOX-3-SENSOR is a versatile accessory, which integrates a Temp&Hum Sensor, a pair of IR Emitter and Receiver, and a Radar Sensor. It empowers user to create sensor network and other sensor applications with ease. The built-in firmware provides a real-time environment Temp&Hum display, a demonstration of human presence monitoring through a 2.4 GHz radar Additionally, it offers an IR learning interface, allowing users to use the ESP-BOX for IR learning of their air conditioning controller, thereby enabling control over the air conditioner. This learning capability also extends to other in-house IR controllers, such as Fan, TV, projector, providing a truly engaging experience.
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/23.png"></td>
<td><img src="./_static/_get_started_static/23.1.png"></td>
</tr>
</table>
### Radar Monitoring
Enter the `Sensor Monitor` screen and tap `ON/OFF` button to enable/ disable radar monitoring feature. When the radar enables switch is set to `ON`, a red body icon will be shown if a person is detected in front of the device, and the icon will turn gray if no person is detected within 2 minutes.
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/25.png"></td>
<td><img src="./_static/_get_started_static/24.png"></td>
</tr>
</table>
### IR Learning
Below the temperature and humidity data and radar functions is the infrared learning module. Currently, this module can only learn the `ON/OFF` function of a remote controller. Please follow the interface instructions to iteratively learn the `ON/OFF` command of your remote controller a total of **four** times. The interface upon successful learning will appear as follows:
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/27.png"></td>
<td><img src="./_static/_get_started_static/28.png"></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/29.png"></td>
<td><img src="./_static/_get_started_static/30.png"></td>
</tr>
</table>
Perform an `ON/OFF` test on your air conditioner by pointing the ESP32-S3-BOX-3-SENSOR towards it. If you find that the logic of the air conditioner's `ON/OFF` function is opposite to your expectations, please click the `Reversal` button below to correct it. Of course, you can also click on `Relearn` to learn other remote controllers.
<div align="center">
<img src="./_static/_get_started_static/31.png">
</div>
>**Tips:**
>* Please be aware that when the ESP32-S3-BOX-3 is not mounted to the ESP32-S3-BOX-3-SENSOR dock, the entire `Sensor Monitor` function will not be operational.
>* During the period of experiencing the built-in firmware with ESP32-S3-BOX-3-DOCK, **avoid hot-plugging** ESP32-S3-BOX-3-DOCK and switching to ESP32-S3-BOX-3-SENSOR. This action could result in ESP32-S3-BOX-3-SENSOR **not being recognized and functioning properly**. To resolve this, simply power again ESP32-S3-BOX-3-SENSOR to restore normal operation.
## Offline Voice Assistant
1. You may say "Hi E. S. P." at any screen to wake up your device. When it wakes up, the screen will display the wake word you just used. If the wake word is not displayed, give it another shot. The screen below indicates your device is listening.
Firstly, connect the RGB LED module to your device. Refer to the pin definitions in the table below, and use Dupont wires to connect the R, G, B, and GND pins of the RGB LED module to the female header connector on the Pmod™ interface of the ESP32-S3-BOX-3-DOCK.
<div align="center">
<img src="./_static/_get_started_static/LED_pin_defination.png">
</div>
1. You may say `"Hi E. S. P." (pronounce in alphabet) ` at any screen to wake up your device. When it wakes up, the screen will display the wake word you just used. If the wake word isn't displayed, give it another try. The screen below indicates your device is listening.
<div align="center">
<img src="./_static/_get_started_static/17.png">
</div>
2. Utter a command within 6 seconds after the beep, such as "turn on the light". You will see the command shown on the screen and the LED light on the module turned on, and hear "OK". Around 6 seconds later you will exit the voice control screen if there is no more command.
2. Utter a command within 6 seconds after the beep, such as `"Turn on the light"`. You will see the command shown on the screen and the LED light on the module turned on, and you will hear an `"OK"`. If no more commands follow in around 6 seconds, you'll exit the voice control screen.
<div align="center">
<img src="./_static/_get_started_static/21.png">
</div>
3. You can use voice commands to enjoy music. Please wake up the device first, then say "play music". The music player will open and start playing built-in music. You can also use voice commands to skip songs or pause music. There are two build-in songs.
3. You can use voice commands to enjoy music. First, wake up the device, then say `"Sing a song"`. This will launch the music player and start playing built-in music. You can also use voice commands to pause or skip songs. There are two built-in songs.
>**Tips:**
>* If the LED light fails to turn on, check whether the module pins are inserted into the right ports.
>* If the BOX recognizes no command within the specified time, you will see `Timeout` and exit the screen in about 1 second.
>* If the BOX doesn't recognize any command within the designated timeframe, a `Timeout` message will appear, and the screen will exit in approximately 1 second.
<div align="center">
<img src="_static/_get_started_static/19.png">
</div>
4. The default commands are: **turn on the light**, **turn off the light**, **turn red**, **turn green**, **turn blue**, **play music**, **next song**, **pause playing**.
4. The default commands are: **turn on the light**, **turn off the light**, **turn red**, **turn green**, **turn blue**, **sing a song**, **next song**, **pause playing**.
## Continuous Speech Recognition
More interestingly, the device supports continuous speech recognition after waking up. This feature makes voice interaction natural and smooth, and brings a human touch to interactive experience.
More interestingly, the device supports continuous speech recognition after waking up. This feature adds a natural and seamless quality to voice interaction, enhancing the interactive experience with a human touch.
**How to use**
- Say "hi, E. S. P" to wake up the device, and you will hear a beep.
- Say your command. If the device recognizes the command, you will hear "OK", and then it will continue to recognize other commands.
- If no command is recognized, the device will wait. If there is not any command in 6 seconds, the device will automatically exit the voice control screen and you need to wake it up again.
- Say `"Hi, E. S. P"` to wake up the device, and you will hear a beep.
- Speak your command. Upon successful recognition, you will hear `"OK"`, and the device will remain ready for further commands.
- If no command is identified, the device will wait. Should no commands be given within 6 seconds, the device will autonomously exit the voice control screen and you need to wake it up again.
**Attention**
- If the device fails to recognize your command for many times, please wait for timeout and wake it up again to use the feature.
- After you say the wake-up word, please do not move the device. Otherwise, the device will fail to recognize your command.
- We recommend voice commands of 3-5 words.
- Currently, the device cannot recognize commands when it plays prompts.
- If your commands are repeatedly unrecognized by the device, allow it to time out and then wake it up again before utilizing the feature.
- Upon saying the wake-up word, please do not move the device, as this could impact command recognition.
- We recommend that voice commands comprise 3-5 words for optimal results.
- Currently, the device cannot recognize commands while it's playing beep.
## Voice Command Customization
The BOX series of development boards are equipped with Espressif proprietary AI Speech Recognition System, which allows you to customize commands through our ESP BOX app. We will take the LED light on the module as an example, to show how to create your own voice commands. For algorithm details, please refer to [Technical Architecture](./technical_architecture.md).
The ESP32-S3-BOX-3 is also equipped with Espressif proprietary AI Speech Recognition System, which allows you to customize commands through our ESP BOX APP. We will take the LED light on the module as an example, to show how to create your own voice commands. For a deeper understanding of the algorithms involved, please refer to the introduction of the ESP-SR speech model in the [Technical Architecture](technical_architecture.md).
**1. Connect to the ESP BOX mobile app**
@@ -158,7 +219,6 @@ The BOX series of development boards are equipped with Espressif proprietary AI
<tr>
<td><img src="_static/_get_started_static/1.jpeg"width="300"/></td>
<td><img src="_static/_get_started_static/2.jpeg"width="300"/></td>
<td><img src="_static/_get_started_static/3.jpeg"width="300"/></td>
</tr>
</table>
@@ -177,6 +237,7 @@ The BOX series of development boards are equipped with Espressif proprietary AI
>**Tips:**
>* Please do not exit the QR code page during the network provisioning.
>* Make sure you connect the device to 2.4 GHz Wi-Fi instead of 5 GHz, and enter the correct Wi-Fi password. If the Wi-Fi password is incorrect, the prompt "Wi-Fi Authentication failed" will pop up.
>* Long press the `Boot` button (i.e. `Funtion` button) for 5 seconds to clear the network information and restore the device to factory settings. After the device is reset, if the QR code or Bluetooth is not working, please restart your device by pressing the `Reset` button.
@@ -186,7 +247,7 @@ The BOX series of development boards are equipped with Espressif proprietary AI
**2. Customize Voice Commands**
2.1. Tap the ESP-BOX device icon and enter the below screen. You can turn on or off the light easily by toggling the button as shown in the picture. You may develop the Fan and Switch features by yourself.
2.1. Select the ESP-BOX device icon and access the screen displayed below. You can conveniently turn the light on or off by simply toggling the button, as indicated in the image. You may need to develop the Fan and Switch functionalities by yourself.
<table border="0" align="center">
<tr>
@@ -204,7 +265,7 @@ The BOX series of development boards are equipped with Espressif proprietary AI
</tr>
</table>
2.3 In the `Configure` tab, you may also customize commands to turn on or off the light and change its color. For example, you can set "good morning" as the command to turn on the light. Click `Save` to return to the previous screen. Then click `Save` again as shown below.
2.3 In the `Configure` tab, you may also customize commands to turn on or off the light and change its color. For example, you can set "Good Morning" as the command to turn on the light. Click `Save` to return to the previous screen. Then click `Save` again as shown below.
<table border="0" align="center">
<tr>
@@ -219,35 +280,35 @@ The BOX series of development boards are equipped with Espressif proprietary AI
<img src="_static/_get_started_static/12.jpeg"width="300" />
</div>
2.5 Now, you can try out your new command! First, say "Hi E. S. P." to wake up your device. Then say "good morning" within 6 seconds to turn on the light. The new command will show on the screen with the module light turned on.
2.5 Now, you can try out your new command! First, say `"Hi E. S. P."` to wake up your device. Then say `"Good Morning"` within 6 seconds to turn on the light. The new command will show on the screen with the module light turned on.
<div align="center">
<img src="_static/_get_started_static/15.png">
</div>
>**To ensure the commands work well, please note:**
>
>* Length of commands: A command should consists of 2-8 words. When creating a series of commands, please try to keep them at similar lengths.
>* Avoid repeating: Please do not include shorter commands in longer commands, or shorter commands will not be recognized. For example, if you create both "turn on" and "turn on the light" commands, "turn on" will not be recognized.
>**For effective command performance, kindly consider the following:**
>* Command Length: A command should consists of 2-8 words. While crafting a sequence of commands, it's advisable to maintain a consistent length across them.
>* Avoid Repetition: Please do not include shorter commands within longer ones. For example, if you create both "turn on" and "turn on the light" commands, the shorter "turn on" command won't be identified.
**3. switch CN and EN voice model**
3.1. Click on your ESP-BOX `About Us` device icon to enter the `About Us` details page. In this page, press `Boot` button, the device will enter `factory Mode` language setting page, and then you can select the voice wake-up model by yourself. After switching the model, the device will reset and restart automatically.
## Switch Voice Model Language
Access the `About Us` details page. In this page, press `Boot` button, the device will enter `factory Mode` voice model language setting page. Here, you can personally select the voice model. After switching the model, the device will reset and restart automatically.
<div align="center">
<img src="_static/_get_started_static/language_select.png"width="300" />
<img src="_static/_get_started_static/22.png"width="300" />
</div>
## FCC Regulations:
## FCC Regulations
This device complies with part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) This device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation. This device has been tested and found to comply with the limits for a Class B digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiated radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause harmful interference to radio or television reception, which can be determined by turning the equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures:
This device complies with part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) This device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation. This device has been tested and found to comply with the limits for a Class B digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause harmful interference to radio or television reception, which can be determined by turning the equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures:
- Reorient or relocate the receiving antenna.
- Increase the separation between the equipment and receiver.
- Connect the equipment into an outlet on a circuit different from that to which the receiver is connected.
- Connect the equipment to an outlet on a circuit different from that to which the receiver is connected.
- Consult the dealer or an experienced radio/TV technician for help.
**FCC Note** Caution: Changes or modifications not expressly approved by the party responsible for compliance could void the users authority to operate the equipment.
**FCC Note Caution:** Changes or modifications not expressly approved by the party responsible for compliance could void the user's authority to operate the equipment.
## RF Exposure Information
+133 -91
View File
@@ -1,64 +1,75 @@
<div align="center">
<img src="./_static/_get_started_static/started_banner.png" width="700px">
</div>
* [English Version](./getting_started.md)
本指南适用于已烧录最新固件的 ESP32-S3-BOX、ESP32-S3-BOX-Lite 产品,下文将其统称为 BOX 系列开发板
本指南适用于 ESP32-S3-BOX-3。对于 ESP32-S3-BOX 和 ESP32-S3-BOX-Lite 的用户指南,请参考最新发布的 [Tag 分支](https://github.com/espressif/esp-box/blob/v0.5.0/docs/getting_started_cn.md)
# 概览
在一个融合创新与艺术、分享与合作至关重要的世界中,我们隆重推出 ESP32-S3-BOX-3,这是一款全开源的 AIoT 开发套件,基于强大的 ESP32-S3 SoC。这款新一代工具为传统开发板带来了积极的改变,它拥有紧凑美观的外壳,并提供丰富的组件,让开发者可以轻松定制和扩展功能,借助 Espressif 的可靠软件解决方案, 如 ESP-BOX、ESP-SR、ESP-Rainmaker 和 ESP-Matter。无论是原型开发还是构建复杂的物联网系统,ESP32-S3-BOX-3 均能提供完美的外形和功能上的平衡,推动着开发板生态的未来发展。
ESP32-S3-BOX-3 出厂固件支持中文和英文离线语音唤醒和命令词识别。ESP-BOX 手机 APP 支持可重配置的 AI 语音交互,您可以创建个性化的命令词来控制智能设备。此外,固件还展示了多种传感器演示,并允许用户进行红外学习,快速将 BOX 转变为空调遥控器。
本用户指南详细介绍了内置固件的基本功能,帮助您开始新的探索之旅。在学习过程中,您将获得开发应用方案的宝贵经验。让我们一起踏上这激动人心的冒险吧!
# 目标群体
包括从事制作、原型设计和开发活动的个人和组织。它旨在满足广泛的用户群体,如业余爱好者、创客、工程师、教育工作者和寻求多功能扩展解决方案的专业人士,以满足他们项目的需求。
# 目标应用
* 智能家居
* 智慧农业
* 智慧城市
* 能源管理
* 物联网原型开发与教育
* 工业物联网
* 智能零售
<div align="center">
<img src="./_static/_get_started_static/application.png" width="700px">
</div>
# 开始使用
搭载 ESP32-S3 芯片的 BOX 系列开发板借助语音助手 + 屏幕控制、传感器、红外控制器和智能 Wi-Fi 网关等功能,为用户提供了一个开发智能家居设备控制系统的平台。BOX 系列开发板出厂支持语音唤醒和离线中英文命令词识别,其 SDK 具备可重构 AI 语音交互功能,用户可以通过自定义命令词控制家居设备。本文仅简要介绍最新固件能够实现的参考功能,供您了解如何开始使用开发板。阅读完该指南,您可以尝试自己开发应用。让我们开始吧!
**BOX 套件包含:**
| ESP32-S3-BOX | ESP32-S3-BOX-Lite |
| :-----: | :---------------------: |
| 主机,可独立使用 | 主机,可独立使用 |
| 供测试的 RGB LED 模块和杜邦线 | 供测试的 RGB LED 模块和杜邦线 |
| Dock 配件,辅助主机立于桌面 | N/A|
**所需硬件:**
请您自行准备一根 USB-C 数据线。
## 连接 RGB LED 模块
参考下面的管脚定义,用杜邦线连接 RGB LED 模块和设备。RGB LED 模块有 R、G、B、GND 四根公头针脚,请将其与 PMOD 1 的 G39、G40、G41、GND 母头端口连接。
## 套件内容
<div align="center">
<img src="./_static/_get_started_static/hardware_pmod.png">
<img src="./_static/_get_started_static/ESP32_S3_BOX_3_KitFigure.png" width="700px">
</div>
- ESP32-S3-BOX-3,独立工作的主模块
- ESP32-S3-BOX-3-DOCK,功能配件,可作为主机的座子
- ESP32-S3-BOX-3-SENSOR,功能配件,展示传感器应用
- ESP32-S3-BOX-3-BRACKET,转接配件,用于将主机安装在其他设备上
- ESP32-S3-BOX-3-BREAD,转接配件,方便主机与标准面包板连接
- 一根 USB-C 数据线
- 一个 RGB LED 模块和杜邦线,用于测试
## 给设备供电
1. 使用 USB-C 数据线给设备供电。
<div align="center">
<img src="./_static/_get_started_static/usb_power.png" width="800px">
<img src="./_static/_get_started_static/usb_power_box.png" width="800px">
</div>
2. 设备启动后,屏幕上将显示 Espressif logo 的开机动画。
<div align="center">
<img src="./_static/_get_started_static/boot_animation.png" width="700px">
<img src="./_static/_get_started_static/boot_animation.png" width="500px">
</div>
## 开始体验
1. 请阅读按键引导页,了解 BOX 系列开发板的按键功能,随后点按 `Next` 进入下一页。
1. 请阅读按键引导页,了解 BOX 开发板的按键功能,随后点按 `Next` 进入下一页。
<table border="0" align="center">
<tr>
<td><b>ESP32-S3-BOX 按键引导页</b></td>
<td><b>ESP32-S3-BOX-Lite 按键引导页</b></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/0.png" width="300px"/></td>
<td><img src="./_static/_get_started_static/1.png" width="300px"/></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/2.png" width="280px"/></td>
<td><img src="./_static/_get_started_static/2.png" width="280px"/></td>
<td><img src="./_static/_get_started_static/0.png"></td>
<td><img src="./_static/_get_started_static/2.png"></td>
</tr>
</table>
2. 请阅读语音助手使用说明引导页,了解 AI 语音功能,随后点按 `OK Let's Go` 进入菜单页。
2. 请阅读语音助手使用说明引导页,了解 AI 语音控制功能,随后点按 `OK Let's Go` 进入页。
<table border="0" align="center">
<tr>
@@ -67,7 +78,7 @@
</tr>
</table>
3. 菜单页有五大功能,分别是 `Device Control``Network``Media Player``Help` 以及 `About Us`,左右滑动可选择功能。比如,进入 `Device Control` 界面,点按 `Light` 可打开或关闭模块上的 LED 灯。回到菜单页,再进入 `Media Player` 界面播放歌曲或调节音量。
3. 主页包含六大功能,分别是 `Sensor Monitor``Device Control``Network``Media Player``Help` 以及 `About Us`,左右滑动可在这些选项之间切换。比如,进入 `Device Control` 界面,点按 `Light` 可打开或关闭灯。回到菜单页,再进入 `Media Player` 界面,可播放音乐或调节系统音量。
<table border="0" align="center">
<tr>
@@ -80,33 +91,78 @@
</tr>
</table>
**仅 ESP32-S3-BOX 支持以下功能:**
4. 按下 ESP32-S3-BOX 顶部的静音按钮之后,语音唤醒和识别功能将被禁用,再次按下则会恢复语音功能。
4. 按下设备顶部的静音按钮以关闭语音唤醒和识别功能,再次按下即可恢复语音功能。
<div align="center">
<img src="_static/_get_started_static/hardware_mute_button.jpg" width="350px">
<img src="./_static/_get_started_static/hardware_mute.png" width="350px">
</div>
5. 轻触屏幕下方的红色圆圈以返回上一页。红圈是一块可自定义的触摸区域,允许用户根据自己的喜好配置额外的触摸功能。
<div align="center">
<img src="./_static/_get_started_static/hardware_home.png" width="350px">
<img src="./_static/_get_started_static/hardware_home.png" width="250px">
</div>
## 传感器监测
此功能需要将 ESP32-S3-BOX-3 安装到 ESP32-S3-BOX-3-SENSOR 底座上。ESP32-S3-BOX-3-SENSOR 是一款多功能配件,集成了温湿度传感器、红外发射和接收器以及雷达传感器。它使用户能够轻松构建传感器网络和其他传感器应用。出厂固件提供实时环境温湿度显示,通过 2.4 GHz 雷达进行人体存在监测。此外,它还提供了红外学习接口,允许用户使用 ESP-BOX 来学习空调遥控器的红外信号,从而实现对空调的控制。这种学习功能还能扩展到其他红外遥控器,如风扇、电视、投影仪,提供别样的互动体验。
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/23.png"></td>
<td><img src="./_static/_get_started_static/23.1.png"></td>
</tr>
</table>
### 雷达监测
进入 `Sensor Monitor`,点击 `ON/OFF`按钮以启用/禁用雷达监测功能。当雷达使能开关设置为`开`时,如果设备前方检测到人体存在,将显示红色人体图标;如果在两分钟内未检测到人体,图标将变为灰色。
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/25.png"></td>
<td><img src="./_static/_get_started_static/24.png"></td>
</tr>
</table>
### 红外学习
在温湿度数据和雷达功能的下方是红外学习模块。目前,此模块只提供了学习遥控器的`开/关`功能。请按照界面指示,连续学习您遥控器的`开/关`命令**四次**。成功学习时的界面显示如下:
<table border="0" align="center">
<tr>
<td><img src="./_static/_get_started_static/27.png"></td>
<td><img src="./_static/_get_started_static/28.png"></td>
</tr>
<tr>
<td><img src="./_static/_get_started_static/29.png"></td>
<td><img src="./_static/_get_started_static/30.png"></td>
</tr>
</table>
此刻,将 ESP32-S3-BOX-3-SENSOR 对准你的空调来进行开关测试,如您发现空调`开/关`功能逻辑与预期相反,请点按下方的 `Reversal` 按钮进行更正。当然,您也可以点击 `Relearn` 来学习其他遥控器。
<div align="center">
<img src="./_static/_get_started_static/31.png">
</div>
>**注意:**
>* 当 ESP32-S3-BOX-3 未安装到 ESP32-S3-BOX-3-SENSOR 底座上时,整个`Sensor Monitor`功能将无法运行。
>* 当使用 ESP32-S3-BOX-3-DOCK 体验出厂固件时,请**避免将 ESP32-S3-BOX-3-DOCK 热插拔**切换到 ESP32-S3-BOX-3-SENSOR。这样的操作会导致 ESP32-S3-BOX-3-SENSOR **无法被识别和正常工作**。若遇到这种情况,只需给 ESP32-S3-BOX-3-SENSOR 重新上下电即可恢复正常运行。
## 离线语音识别
1. 您可以在任意界面使用唤醒词来唤醒设备,唤醒词为“Hi 乐鑫”。设备唤醒后,唤醒词将显示在屏幕上并伴有提示音“我在”。如果唤醒词未出现,请再次尝试。出现如下界面表示您的设备正在倾听
首先,参考下表的管脚定义,将 RGB LED 模块的 R、G、B、GND 四个引脚用杜邦线连接至 ESP32-S3-BOX-3-DOCK PMOD 接口的母头连接器上
<div align="center">
<img src="./_static/_get_started_static/LED_pin_defination.png">
</div>
1. 您可以在任意界面使用唤醒词来唤醒设备,唤醒词为`“Hi 乐鑫”`。设备唤醒后,唤醒词将显示在屏幕上并伴有提示音`“我在”`。如果唤醒词未出现,请再次尝试。出现如下界面表示您的设备正在倾听。
<div align="center">
<img src="./_static/_get_started_static/16.png">
</div>
2. 请在提示音响起后的 6 秒内对设备说出命令词,比如“打开电灯”。您将看到命令词显示在屏幕上,模块的 LED 灯亮起,同时听到语音反馈“好的”。若无其他命令词,设备将在 6 秒左右退出语音控制界面。
2. 请在提示音响起后的 6 秒内对设备说出命令词,比如`“打开电灯”`。您将看到命令词显示在屏幕上,RGB LED 模块的灯亮起,同时听到语音反馈`“好的”`。若无其他命令词,设备将在 6 秒左右退出语音控制界面。
<div align="center">
<img src="./_static/_get_started_static/20.png">
</div>
3. 您还可以使用语音命令词欣赏音乐。请先唤醒设备,然后说出命令词“播放音乐”,设备将打开音乐播放器并开始播放内置音乐。您也可以用语音命令词切换歌曲或者暂停播放。最新固件默认自带两歌曲。
3. 您还可以使用语音命令词欣赏音乐。请先唤醒设备,然后说出命令词`“播放音乐”`,设备将打开音乐播放器并开始播放内置音乐。您也可以用语音命令词切换歌曲或者暂停播放。出厂固件默认自带两歌曲。
>**注意:**
>
@@ -114,7 +170,7 @@
>* 若在规定时间内未识别到命令词,则会显示 `超时`,约 1 秒后界面关闭。
<div align="center">
<img src="_static/_get_started_static/19.png">
<img src="_static/_get_started_static/18.png">
</div>
4. 默认命令词包括:**打开电灯**,**关闭电灯**,**调成红色**,**调成绿色**,**调成蓝色**,**播放音乐**,**切歌**,**暂停播放**。
@@ -125,50 +181,39 @@
**使用方式**
- 使用“Hi,乐鑫”唤醒设备,唤醒后设备播放提示音“我在”。
- 说出命令词。如设备识别出命令词,则会播放提示音“好的”,然后将继续识别其他命令词。
- 使用`“Hi,乐鑫”`唤醒设备,唤醒后设备播放提示音`“我在”`
- 说出命令词。如设备识别出命令词,则会播放提示音`“好的”`,然后将继续识别其他命令词。
- 如设备未识别命令词则会继续等待,6 秒内没有新命令将自动退出语音控制界面,使用语音识别功能需要再次唤醒设备。
**注意**
- 如设备多次未能识别命令词,请再次唤醒重新识别
- 说出唤醒词后请勿移动设备,设备不支持在移动中识别命令词
- 我们推荐使用长度为 3-5 个汉字的命令词
- 暂不支持播放提示音时识别命令词
- 如设备多次未能识别命令词,请再次唤醒重新识别
- 说出唤醒词后请勿移动设备,设备不支持在移动中识别命令词
- 我们推荐使用长度为 3-5 个汉字的命令词
- 暂不支持播放提示音时识别命令词
**额外支持的命令词**
**中文模型额外支持的命令词**
| command ID | 命令词 | command ID | 命令词 | command ID | 命令词 |
| :--------: | :----------: | :--------: | :----------: | :--------: | :------------: |
| 0 | 打开空调 | 24 | 二十六度 | 48 | 音量调到最小 |
| 1 | 关闭空调 | 25 | 二十七度 | 49 | 打开电灯 |
| 2 | 舒适模式 | 26 | 二十八度 | 50 | 关闭电灯 |
| 3 | 制冷模式 | 27 | 二十九度 | 51 | 打开卧室灯 |
| 4 | 制热模式 | 28 | 三十度 | 52 | 关闭卧室灯 |
| 5 | 加热模式 | 29 | 增大风速 | 53 | 打开客厅灯 |
| 6 | 除湿模式 | 30 | 减小风速 | 54 | 关闭客厅灯 |
| 7 | 送风模式 | 31 | 高速风 | 55 | 打开音箱 |
| 8 | 升高温度 | 32 | 中速风 | 56 | 关闭音箱 |
| 9 | 降低温度 | 33 | 低速风 | 57 | 拉开窗帘 |
| 10 | 温度调到最高 | 34 | 自动风 | 58 | 关闭窗帘 |
| 11 | 温度调到最低 | 35 | 打开左右摆风 | 59 | 打开风扇 |
| 12 | 太热了 | 36 | 打开上下摆风 | 60 | 关闭风扇 |
| 13 | 太冷了 | 37 | 关闭左右摆风 | 61 | 打开饮水机插座 |
| 14 | 十六度 | 38 | 关闭上下摆风 | 62 | 关闭饮水机插座 |
| 15 | 十七度 | 39 | 打开辅热 | 63 | 打开投影仪 |
| 16 | 十八度 | 40 | 关闭辅热 | 64 | 关闭投影仪 |
| 17 | 十九度 | 41 | 打开自清洁 | 65 | 灯光调暗 |
| 18 | 二十度 | 42 | 关闭自清洁 | 66 | 灯光调亮 |
| 19 | 二十一度 | 43 | 打开静音 | | |
| 20 | 二十二度 | 44 | 关闭静音 | | |
| 21 | 二十三度 | 45 | 音量大点 | | |
| 22 | 二十四度 | 46 | 音量小点 | | |
| 23 | 二十五度 | 47 | 音量调到最大 | | |
| command ID | 命令词 |
| :--------: | :----------:|
| 0 | 打开空调 |
| 1 | 关闭空调 |
| 2 | 舒适模式 |
| 3 | 制冷模式 |
| 4 | 制热模式 |
| 5 | 加热模式 |
| 6 | 除湿模式 |
| 7 | 送风模式 |
| 8 | 升高温度 |
| 9 | 降低温度 |
> 上述命令词仅供演示,由于没有接入相关智能设备,所以无法实现命令词功能。
>**注意:**
>
> 上述命令词仅供演示,您可以自行开发接入相关智能设备,用语音识别实现真实的控制效果。
## 语音命令词自定义
BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过 ESP BOX 应用程序自定义任意命令词。我们将以模块上的 LED 灯为例,介绍如何自定义语音命令词。如需了解算法详细信息,请参阅[技术架构](./technical_architecture_cn.md)。
ESP32-S3-BOX-3 还配备了乐鑫专有的 AI 语音识别系统,通过我们的 ESP BOX APP,您可以自定义命令词。我们将以模块上的 LED 灯为例,展示如何创建您自己的语音命令词。如需深入了解所涉及的算法,请参考[技术架构](technical_architecture_cn.md)内对 ESP-SR 语音模型的介绍。
**1. 连接 ESP BOX 手机应用程序**
@@ -182,7 +227,7 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
<img src="_static/_get_started_static/Picture1.png" width="100 px">
</div>
1.2. 如您第一次使用此应用程序,请先注册账
1.2. 如您第一次使用此应用程序,请先注册账
1.3. 登录您的 ESP BOX 账号并打开手机蓝牙。点按界面下方 `+`,扫描设备上的二维码配置网络。
@@ -190,7 +235,6 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
<tr>
<td><img src="_static/_get_started_static/1.jpeg"width="300"/></td>
<td><img src="_static/_get_started_static/2.jpeg"width="300"/></td>
<td><img src="_static/_get_started_static/3.jpeg"width="300"/></td>
</tr>
</table>
@@ -207,9 +251,9 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
<img src="_static/_get_started_static/7.png">
</div>
>**注意:**
>
>* 配网期间,请不要退出二维码界面。
>* 请确保设备接入 2.4 GHz Wi-Fi 而非 5 GHz,且 Wi-Fi 密码输入正确。如果 Wi-Fi 密码错误,设备将会提示 "Wi-Fi Authentication failed"。
>* 长按 `Boot` 键(即 `Function` 键)5 秒可清除网络配置信息,恢复出厂设置。设备重置后,如遇二维码扫码失败或蓝牙搜索不到设备,请按 `Reset` 键重启设备。
@@ -217,7 +261,7 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
<img src="_static/_get_started_static/7.jpeg"width="300" />
</div>
**2. 自定义语音命令词**
**2. 自定义语音命令词**
2.1. 点按您的 ESP-BOX 设备图标,进入如下界面。在此界面,您可轻松控制开灯或者关灯。此外,我们预留了 `Switch``Fan` 功能供您自行开发。
@@ -253,7 +297,7 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
<img src="_static/_get_started_static/12.jpeg"width="300" />
</div>
2.5. 现在,您可以体验新命令词了!请先使用“Hi 乐鑫”唤醒设备,然后在 6 秒内说出打开电灯的新命令词“早上好”,新命令词将显示在屏幕上,同时模块上的 LED 灯会亮起。
2.5. 现在,您可以体验新命令词了!请先使用`“Hi 乐鑫”`唤醒设备,然后在 6 秒内说出打开电灯的新命令词`“早上好”`,新命令词将显示在屏幕上,同时模块上的 LED 灯会亮起。
<div align="center">
<img src="_static/_get_started_static/14.png"width="300" />
@@ -264,13 +308,11 @@ BOX 系列开发板配备乐鑫专有的 AI 语音识别系统,您可以通过
>* 命令词长度:命令词长度需要在 2-8 个汉字之间。在定义一系列命令词时,最好可以保持不同命令词的长度相似。
>* 避免重复:请勿在长命令词中包含短命令词,否则短命令词将无法识别。比如“打开”和“打开灯”,设备将无法识别“打开”。
**3. 中英文语音模型切换**
## 中英文语音模型切换
3.1. 点按您的 ESP-BOX `About Us` 设备图标,进入 `About Us` 详情页。在此界面,按下 `Boot` 按键,设备会快速进入 `factory Mode` 语言设置页面,用户可自行设置语音的唤醒模型。模型切换后设备将自动重置重启。
点按您的 ESP-BOX `About Us` 设备图标,进入 `About Us` 详情页。在此界面,按下 `Boot` 按键,设备会快速进入 `factory Mode` AI 语音模型语言设置页面,用户可自行选择偏好的模型语言。模型切换后设备将自动重置重启。
<div align="center">
<img src="_static/_get_started_static/language_select.png"width="300" />
<img src="_static/_get_started_static/22.png"width="300" />
</div>
此指南仅供您简要了解如何在 BOX 系列开发板上使用最新的固件。现在,请开始编写自己的应用程序,开启物联网之旅吧!
@@ -0,0 +1,95 @@
* [中文版本](./hardware_overview_for_box_3_cn.md)
## Hardware Overview
### ESP32-S3-BOX-3
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp32_s3_box_3.png" width="300px">
</div>
ESP32-S3-BOX-3 is the main unit powered by the ESP32-S3-WROOM-1 module, which offers 2.4 GHz Wi-Fi + Bluetooth 5 (LE) wireless capability as well as AI acceleration capabilities. On top of 512 KB SRAM provided by the ESP32-S3 SoC, the module comes with additional 16 MB Quad flash and 16 MB Octal PSRAM. The board is equipped a 2.4-inch 320 x 240 SPI touch screen (the red circle supports touch), two digital microphones, a speaker, 3axis Gyroscope, 3axis Accelerometer, one Type-C port for power and download/debug, a high-density PCIe connector which allows for hardware extensibility, as well as three functional buttons.
#### Orthographic View
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_overview.png" width="700px">
</div>
#### Technical Specification
![specs](../../_static/box_3_hardware_overview/specs_for_box_3.png)
### ESP32-S3-BOX-3-DOCK
<div align="center">
<img src="./../../_static/box_3_hardware_overview/DOCK.jpg" width="300px">
</div>
ESP32-S3-BOX-3-DOCK is designed to serve as a stand for ESP32-S3-BOX-3 through the gold fingers, and offers diverse expandability. It has two Pmod™ compatible headers, which allow users to expand additional peripheral modules. These headers offer 16 programmable GPIOs that can supply 3.3 V power to peripherals. One USB Type-A port for connecting to diverse USB devices such as USB camera (with a maximum resolution of 720 P), USB disk, and other HID devices. While the another USB Type-C port for 5 V input power only.
#### Orthographic View
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_dock_overview.png" width="700px">
</div>
#### Technical Specification
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_dock.png)
* [Digilent Pmod™ Interface Specification](https://digilent.com/reference/_media/reference/pmod/pmod-interface-specification-1_3_1.pdf)
#### Pinout Diagram
<div align="center">
<img src="./../../_static/box_3_hardware_overview/pinlayout_box_3_dock.png" width="700px">
</div>
### ESP32-S3-BOX-3-SENSOR
<div align="center">
<img src="./../../_static/box_3_hardware_overview/SENSOR.jpg" width="300px">
</div>
ESP32-S3-BOX-3-SENSOR is a versatile accessory, which integrates Temp&Hum Sensor, IR Emitter and Receiver, Radar Sensor, 18650 Rechargeable Battery slot and MicroSD Card slot. It empowers user to create wide range of innovative projects with ease. Integrate multiple sensors for detection and control, leverage the rechargeable battery for portability, and expand your storage capabilities using the MicroSD Card slot (Expand storage up to 32 GB).
#### Orthographic View
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_sensor_overview.png" width="700px">
</div>
#### Technical Specification
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_sensor.png)
>**Tips:**
>* When charging the 18650 battery, please switch the toggle to 'ON' and plug the ESP32-S3-BOX-3 into the ESP32-S3-BOX-3-SENSOR.
### ESP32-S3-BOX-3-BRACKET
<div align="center">
<img src="./../../_static/box_3_hardware_overview/BRACKET.jpg" width="300px">
</div>
ESP32-S3-BOX-3-BRACKET can be utilized to help ESP32-S3-BOX-3 mount to other devices, opens up a multitude of possibilities for transforming non-smart devices into smart ones. By leveraging its two Pmod™ compatible headers, user can effectively integrate wireless connectivity, voice control and screen control capabilities. The ESP32-S3-BOX-3-BRACKET adapter empowers you to unleash the full potential of your non-smart devices.
#### Orthographic View
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_bracket_overview.png" width="700px">
</div>
#### Technical Specification
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_bracket.png)
### ESP32-S3-BOX-3-BREAD
<div align="center">
<img src="./../../_static/box_3_hardware_overview/BREAD.jpg" width="300px">
</div>
ESP32-S3-BOX-3-BREAD is an adapter that enables easy connection of the ESP32-S3-BOX-3 to a standard breadboard, providing convenience for makers who enjoy hands-on projects to expand and connect with other devices. It utilizes a high-density PCIe connector and two rows of 2.54 mm pitch pins to expose the ESP32-S3's 16 programmable GPIOs, making ESP32-S3-BOX-3-BREAD highly practical and functional.
#### Orthographic View
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_bread_overview.png" width="700px">
</div>
#### Technical Specification
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_bread.png)
#### Pinout Diagram
<div align="center">
<img src="./../../_static/box_3_hardware_overview/pinlayout_box_3_bread.png" width="700px">
</div>
## Hardware Source Files
* [ESP32-S3-BOX-3 Kit PCB](../../../hardware/PCB_ESP32-S3-BOX-3_V1.0)
* [ESP32-S3-BOX-3 Kit Schematic](../../../hardware/SCH_ESP32-S3-BOX-3_V1.0)
* [ESP32-S3-BOX-3 3D Shell CAD](../../../hardware/esp32_s3_box_3_shell_stl)
@@ -0,0 +1,100 @@
* [English Version](./hardware_overview_for_box_3.md)
## 硬件概览
### ESP32-S3-BOX-3
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp32_s3_box_3.png" width="300px">
</div>
ESP32-S3-BOX-3 主控单元由 ESP32-S3-WROOM-1 模组驱动,该模块提供了 2.4 GHz Wi-Fi + Bluetooth 5 (LE) 的无线功能,同时具备 AI 加速能力。在 ESP32-S3 SoC 提供的 512 KB SRAM 基础上,该模块还附带额外的 16 MB 四线 Flash 和 16 MB 八线 PSRAM。开发板配备一片 2.4 英寸 320 x 240 的 SPI 接口触摸屏(‘红圈’支持触摸),两个数字麦克风,一个扬声器,三轴陀螺仪,三轴加速度计,一个用于电源和下载/调试的 Type-C 接口,一个可实现硬件扩展的高密度 PCIe 连接器,以及三个功能按钮。
#### 正视图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_overview.png" width="700px">
</div>
#### 规格参数
![specs](../../_static/box_3_hardware_overview/specs_for_box_3.png)
### ESP32-S3-BOX-3-DOCK
<div align="center">
<img src="./../../_static/box_3_hardware_overview/DOCK.jpg" width="300px">
</div>
ESP32-S3-BOX-3-DOCK 被设计成通过插拔金手指为 ESP32-S3-BOX-3 提供支架,并拥有多样化的扩展性。它有两个 Pmod™ 兼容的 I/O 接口,允许用户扩展额外的外设模块。这些接口共提供了 16 个可编程的 GPIO 引脚,同时可以向外设供 3.3 V 电。一个 USB Type-A 接口用于连接各种 USB 设备,如 USB 摄像头(最大分辨率为 720 P)、U 盘和其他 HID 设备。另一个 USB Type-C 接口仅用于 5 V 输入电源。
#### 正视图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_dock_overview.png" width="700px">
</div>
#### 规格参数
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_dock.png)
* [Digilent Pmod™ Interface Specification](https://digilent.com/reference/_media/reference/pmod/pmod-interface-specification-1_3_1.pdf)
#### 引脚图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/pinlayout_box_3_dock.png" width="700px">
</div>
### ESP32-S3-BOX-3-SENSOR
<div align="center">
<img src="./../../_static/box_3_hardware_overview/SENSOR.jpg" width="300px">
</div>
ESP32-S3-BOX-3-SENSOR 是一个多功能配件,集成了温湿度传感器、红外发射器和接收器、雷达传感器、18650 可充电电池插槽和 MicroSD 卡插槽。它使用户能够轻松地创建各种创新项目。通过整合多个传感器进行检测和控制,利用可充电电池实现便携性,并使用 MicroSD 卡插槽扩展存储能力(最多可扩展到 32 GB)。
#### 正视图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_sensor_overview.png" width="700px">
</div>
#### 规格参数
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_sensor.png)
>**注意**
>* 当给 18650 电池充电时,请将开关拨至“ON”位置,并将 ESP32-S3-BOX-3 插入 ESP32-S3-BOX-3-SENSOR。
### ESP32-S3-BOX-3-BRACKET
<div align="center">
<img src="./../../_static/box_3_hardware_overview/BRACKET.jpg" width="300px">
</div>
ESP32-S3-BOX-3-BRACKET 可用于帮助 ESP32-S3-BOX-3 安装到其他设备上,为将非智能设备转变为智能设备开启了多种可能性。通过利用其两个 Pmod™ 兼容的引脚,用户可以有效地为非智能设备集成无线连接、语音控制和屏幕控制功能。ESP32-S3-BOX-3-BRACKET 转接配件使您能够充分发挥非智能设备的潜力。
#### 正视图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_bracket_overview.png" width="700px">
</div>
#### 规格参数
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_bracket.png)
### ESP32-S3-BOX-3-BREAD
<div align="center">
<img src="./../../_static/box_3_hardware_overview/BREAD.jpg" width="300px">
</div>
ESP32-S3-BOX-3-BREAD 是一个转接板,方便用户轻松连接 ESP32-S3-BOX-3 到标准面包板,为热爱自己动手的创客们扩展和连接其他设备提供了便利。它利用高密度 PCIe 连接器和两排 2.54 mm 间距的排针,引出了 ESP32-S3 共 16 个可编程 GPIO 引脚,ESP32-S3-BOX-3-BREAD 极具实用价值。
#### 正视图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/esp_box_3_bread_overview.png" width="700px">
</div>
#### 规格参数
![specs](../../_static/box_3_hardware_overview/sepc_for_box_3_bread.png)
#### 引脚图
<div align="center">
<img src="./../../_static/box_3_hardware_overview/pinlayout_box_3_bread.png" width="700px">
</div>
## 硬件开源资料
* [ESP32-S3-BOX-3 Kit PCB](../../../hardware/PCB_ESP32-S3-BOX-3_V1.0)
* [ESP32-S3-BOX-3 Kit Schematic](../../../hardware/SCH_ESP32-S3-BOX-3_V1.0)
* [ESP32-S3-BOX-3 3D Shell CAD](../../../hardware/esp32_s3_box_3_shell_stl)
+3 -3
View File
@@ -24,9 +24,9 @@ In addition to system-level support, ESP-BOX also integrates several excellent s
ESP-SR is a speech recognition framework developed by Espressif for the field of intelligent speech. ESP-SR can provide high-performance and high-reliability speech recognition functions for developers by using ESP32-S3 AI extension instructions. Now, this framework has three modules:
1. Espressif's acoustic algorithm ([Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/acoustic_algorithm)), which is specially designed to improve speech recognition performance in far-field or noisy environments to enable developers to obtain high-quality and stable audio data. It mainly includes AEC (Acoustic Echo Cancellation), BSS (Blind Source Separation), NS (Noise Suppression). ESP-SR encapsulates the above algorithms into simple APIs. Without undersatanding details of these algorithms, developers only need to arrange and combine the algorithms that need to be used for specific application scenarios, and input the voice data to be processed according to the API format requirements and can get the results.
2. Espressif wake-up word engine ([WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)), which is used to monitor specific language fragments in real-time in a continuous audio streaming, and wake up the smart device from the standby state to the voice interaction state. WakeNet uses MFC (Mel-frequency Cepstrum) to obtain the features of the input audio clip, then classifies the characteristic signals by the neural network algorithm optimized for ESP32-S3. That can trigger the wake-up signal when analyzed the command word, and the [effective recognition rate](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test) is more than 80% in a noisy environment. ESP-BOX provides different offline wake words like "Hi, ESP" or "Hi, Lexin". And also allows [customized wake word](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/ESP_Wake_Words_Customization.md).
3. Espressif multi-command recognization model ([MultiNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md)), which is designed to provide a flexible command recognition after the device is awakened. MultiNet supports both Chinese and English speech commands, like "Guan Bi Dian Deng", "Turn on the air conditioner" through a lightweight model specially designed based on CRNN and CTC, and supports up to 200 offline customized speech commands. The command words can be added and modified simply through configuring [Chinese Pinyin string](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) or [English Phonetic String](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) and the corresponding ID. Developers do not need to understand the details of the MultiNet nor re-train the model. the event will be triggered after the command words are recognized.
1. Espressif's acoustic algorithm ([Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/acoustic_algorithm)), which is specially designed to improve speech recognition performance in far-field or noisy environments to enable developers to obtain high-quality and stable audio data. It mainly includes AEC (Acoustic Echo Cancellation), BSS (Blind Source Separation), NS (Noise Suppression). ESP-SR encapsulates the above algorithms into simple APIs. Without understanding details of these algorithms, developers only need to arrange and combine the algorithms that need to be used for specific application scenarios, and input the voice data to be processed according to the API format requirements and can get the results.
2. Espressif wake-up word engine ([WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)), which is used to monitor specific language fragments in real-time in a continuous audio streaming, and wake up the smart device from the standby state to the voice interaction state. WakeNet uses MFC (Mel-frequency Cepstrum) to obtain the features of the input audio clip, then classifies the characteristic signals by the neural network algorithm optimized for ESP32-S3. That can trigger the wake-up signal when analysed the command word, and the [effective recognition rate](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test) is more than 80% in a noisy environment. ESP-BOX provides different offline wake words like "Hi, ESP" or "Hi, Lexin". And also allows [customized wake word](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/ESP_Wake_Words_Customization.md).
3. Espressif multi-command recognition model ([MultiNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md)), which is designed to provide a flexible command recognition after the device is awakened. MultiNet supports both Chinese and English speech commands, like "Guan Bi Dian Deng", "Turn on the air conditioner" through a lightweight model specially designed based on CRNN and CTC, and supports up to 200 offline customized speech commands. The command words can be added and modified simply through configuring [Chinese Pinyin string](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) or [English Phonetic String](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands) and the corresponding ID. Developers do not need to understand the details of the MultiNet nor re-train the model. The event will be triggered after the command words are recognized.
**ESP-HMI**
+1 -1
View File
@@ -24,7 +24,7 @@ ESP-IDF 是乐鑫官方开源操作系统框架,同时兼容 ESP32、ESP32-S
ESP-SR 是乐鑫面向智能语音领域开发的语音识别框架,该框架可利用 ESP32-S3 AI 扩展指令进行运算加速,为开发者提供开箱即用的高性能、高可靠性的语音识别功能,它包括:
1. 声学前端([Audio Front-End, AFE](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/acoustic_algorithm))算法集,用于提升智能语音设备在远场噪声环境中的交互能力,使开发者获得高质量且稳定的音频数据。它主要包括可以有效滤除扬声器回声的回声消除算法(Acoustic Echo CancellationAEC), 用于强化麦克风阵列方向性的盲源分离算法(Blind Source SeparationBSS),和用于处理环境噪声的噪声抑制算法(Noise SuppressionNS)。ESP-SR 将以上算法封装成简单的 API,开发者无需关心这些算法的具体细节,只需要针对具体的应用场景,对需要使用的算法排列组合,并按照 API 的格式要求输入待处理的语音数据,即可得到声学前端算法的处理结果。
2. 唤醒词识别模型([WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)),用于在连续的音频流中实时监测出特定的语言片段,将智能设备从待机状态唤醒至语音交互状态。WakeNet 通过计算音频片段的梅尔倒频谱(Mel-Frequency Cepstrum, MFC)作为输入特征,使用针对 ESP32-S3 优化的神经网络算法对特征信号进行分类,在监测到命令词时触发唤醒信号,可在噪声环境下获得不小于 80% 的[有效识别率](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test)。ESP-BOX 提供了开箱即用的离线唤醒词 “Hi, ESP” 和 “Hi, 乐鑫”,开发者无需额外的开发工作即可使用,同时乐鑫也支持为开发者[定制独特的唤醒词](https://github.com/espressif/esp-sr/blob/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/%E4%B9%90%E9%91%AB%E8%AF%AD%E9%9F%B3%E5%94%A4%E9%86%92%E8%AF%8D%E5%AE%9A%E5%88%B6%E6%B5%81%E7%A8%8B.md)。
2. 唤醒词识别模型([WakeNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md)),用于在连续的音频流中实时监测出特定的语言片段,将智能设备从待机状态唤醒至语音交互状态。WakeNet 通过计算音频片段的梅尔倒频谱(Mel-Frequency Cepstrum, MFC)作为输入特征,使用针对 ESP32-S3 优化的神经网络算法对特征信号进行分类,在监测到命令词时触发唤醒信号,可在噪声环境下获得不小于 80% 的[有效识别率](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/README.md#performance-test)。ESP-BOX 提供了开箱即用的离线唤醒词 “Hi, ESP” 和 “Hi, 乐鑫”,开发者无需额外的开发工作即可使用,同时乐鑫也支持为开发者[定制独特的唤醒词](https://github.com/espressif/esp-sr/blob/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/wake_word_engine/乐鑫语音唤醒词定制流程.md)。
3. 命令词识别模型([MultiNet](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md)),用于在设备唤醒后,识别用户特定的中英文语音命令,例如 “Turn on the air conditioner”,“关闭电灯” 等。MultiNet 通过基于 CRNN 和 CTC 的卷积循环神经网络,对输入音频片段进行多命令词识别,可同时支持最多 200 个离线命令词。 开发者不需要了解识别模型细节,也无需重新进行模型训练,通过简单配置[中文拼音字符串](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands)或[英文音标字符串](https://github.com/espressif/esp-sr/tree/3ce34fe340af15e3bfb354c21c1ec2e6e31a37e8/docs/speech_command_recognition/README.md#modify-speech-commands)以及对应 ID,即可添加或修改命令词,模型在识别到语音命令词后将触发包含 ID 信息的事件。
**ESP-HMI**
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
In this example, we are utilizing the OpenAI API in conjunction with an ESP-BOX to create a voice-based chatbot. The ESP-BOX is a device or system that incorporates an ESP32-S3 microcontroller. The purpose of this implementation is to enable users to communicate with the chatbot using spoken language. The process involves capturing audio input from the user, sending it to the OpenAI API for processing, and receiving a response that is then converted into speech and played back to the user.
+7 -5
View File
@@ -2,20 +2,22 @@
| Board | Support Status |
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX | NO |
| ESP32-S3-BOX-Lite | NO |
| ESP32-S3-BOX-3 | YES |
The factory_demo is a built-in firmware of ESP-BOX series products, which integrates LVGL, ESP-Rainmaker and ESP-SR.
The factory_demo is a built-in firmware of ESP32-S3-BOX-3, which integrates LVGL, ESP-Rainmaker and ESP-SR. For ESP32-S3-BOX and ESP32-S3-BOX-Lite, please refer to the [latest release tag branch](https://github.com/espressif/esp-box/tree/v0.5.0/examples/factory_demo). The master branch will gradually offer limited support for these two development boards.
## How to use example
<font color="red">[Note]: </font>This example requires ESP-IDF release/v5.1 or later.
Please check the [User-Guide](../../docs/getting_started.md).
Please check the [User Guide](../../docs/getting_started.md) for more details about how to use the demo.
### Hardware Required
* A ESP32-S3-Box or ESP32-S3-BOX-Lite
* A ESP32-S3-BOX-3
* An USB-C cable for power supply and programming
### Build and Flash
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
Display PNG image with LVGL.
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | NO |
| ESP32-S3-BOX-3 | NO |
LVGL official demos.
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
This example creates a 3 channel matter On/Off Switch device on esp32s3 box platform.
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
Play MP3 music on ESP-BOX.
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
This demo makes ESP32-S3-BOX work as a USB headset that can play music and record voice. Default uses 48k single mic and single channel audio data, and it uses FFT to display the music spectrum on the screen and supports multiple animations.
+1
View File
@@ -4,6 +4,7 @@
| ----------------- | -------------- |
| ESP32-S3-BOX | YES |
| ESP32-S3-BOX-Lite | YES |
| ESP32-S3-BOX-3 | NO |
The example is a prototype that simulates a smart watering system. The example utilizes ESP-BOX as a control panel, providing cloud connection capability as well as a voice control function. It also supports remote control from a mobile app using ESP RainMaker.