Voice assistants are everywhere, but most of them are rigid. Say exactly the right phrase or they fail. I wanted to build something different — a system that understands intent, not just keywords.
Architecture Overview
The system runs on a Raspberry Pi 4 as the central hub. A microphone captures voice input, which is transcribed locally using Whisper. The transcript is then passed to an LLM (via API) that extracts the intent and maps it to a device command.
- Raspberry Pi 4 — Central controller
- OpenAI Whisper — Local speech-to-text
- LLM API — Natural language intent extraction
- MQTT — Device communication protocol
- ESP8266 modules — Relay controllers for devices
The Intent Layer
The interesting part is how the LLM understands natural language commands. I prompt it with the list of available devices and ask it to return a structured JSON action — no free-form text, just machine-readable output.
{
"device": "living_room_lights",
"action": "turn_on",
"params": { "brightness": 70 }
}What I Learned
Prompt engineering is more engineering than art. Getting consistent, structured outputs from an LLM requires careful constraint design. I also learned that MQTT is an incredibly elegant protocol for IoT — lightweight, reliable, and perfect for embedded systems.