硬件接入AI最简单、最低成本的方案
The simplest and lowest cost solution for any item to access AI
Provides a complete AI dialogue solution for your development board, including but not limited to the
IAT(ASR)+LLM+TTS
integration solution for theESP32
series development boards. It is injected into the project as a dependency without affecting existing projects.
For developing the dialogue functionality of robots, you only need to prepare the IAT(ASR)
, LLM
, and TTS
services, and leave the rest to ESP-AI
.
The server-side code of this project is based on Node.js
, and the hardware code is based on Arduino
/IDF
.
Open source is not easy, click the Star
button in the upper right corner to show your support~
- ✔️ Customizable offline wake words with multiple built-in wake-up methods (voice, button, serial port, Tianwen ASRPro)
- ✔️ Complete conversation chain: IAT (ASR) ➡️ LLM/RAG ➡️ TTS
- ✔️ Fast response algorithms for TTS/LLM, designed to balance service cost while providing the quickest response time
- ✔️ Supports conversation interruption
- ✔️ Recognizes user commands (appliance control, singing, etc.) and can dynamically respond based on context
- ✔️ Configurable
- ✔️ Plugin-based, allowing integration with any LLM/TTS/IAT using plugins
- ✔️ One-to-many relationship between service and clients, with independent configuration for each client (hardware)
- ✔️ Connection supports authentication
- ✔️ Full-chain streaming data interaction
- ✔️ Developer platform offers: free services, visual configuration, etc.
- ✔️ Client configuration webpage provided
- ✔️ Easily handles high concurrency scenarios (requires Nginx for load balancing)
- ✔️ Ready to use out of the box
- [ ] 🤔 Improve accuracy of built-in offline wake-up (currently recommended to use Tianwen ASRPro)
- [ ] 🤔 Online wake word generation
- [ ] 🤔 Develop plugins in other languages (to avoid relying solely on Node.js for plugin development)
- [ ] 🤔 Provide a dedicated development board (to avoid current complex wiring)
docker run -itd -p 8080:8080 -v /esp-ai-server/index.js:/server/index.js --name esp-ai-server registry.cn-shanghai.aliyuncs.com/xiaomingio/esp-ai:1.0.0
Download the dependency on the release page and burn it to the development board, see details: Client Install
Below are the Node.js
and Arduino
codes you need to write if you only require dialogue functionality.
For commercial use, please purchase a commercial license. For more details, see Commercial License.
The commercial license is granted per unit of Company Name + Product Name + Version, for example: "Shenzhen XXX Company Smart Doll V1.0".
The commercial license fee for a single product version is 10,000 RMB, payable in a one-time payment.
License inquiry/report address: https://espai.fun/commercial-license.html.
QQ Discussion Group: 854445223
Let's build a better esp-ai
together.
We warmly invite contributions from everyone. Feel free to share your ideas through Pull Requests or GitHub Issues.
|
|
|
If this project has helped your research, please cite us:
@software{ESP-AI,
title = {{ESP-AI}},
author = {小明IO},
year = 2024,
journal = {GitHub repository},
publisher = {GitHub},
howpublished = {\url{https://github.com/wangzongming/esp-ai}}
}