Show HN: Ved AI Voice Assistant 展示 HN：Ved AI 语音助手

TL;DR

• Ved is a resource-constrained, fully offline AI agent designed for Windows, Linux, and macOS, requiring as little as 12 GB VRAM and 32 GB RAM for local operation.
• The architecture utilizes a LangGraph orchestrator with distinct lanes: a lightweight 3B model for casual chat, and a planner-executor duo (8B/7B planner + 3B executor) for complex tasks involving tools.
• Key features include a Tkinter desktop UI, an offline voice pipeline using faster-whisper and Piper TTS, per-thread RAG with FIFO eviction, and a script executor for uploaded Python files.
• The system supports deterministic model switching between standard, turbo, coder, and hibernate modes, with an optional fallback to cloud APIs like OpenRouter for lower-end hardware.
• Privacy is prioritized with local-only telemetry and human-in-the-loop approval mechanisms for sensitive filesystem and code execution tool calls.

Why It Matters

This project demonstrates a practical approach to deploying sophisticated AI agents on consumer-grade hardware without relying on expensive cloud infrastructure, making advanced AI capabilities accessible to individuals and small teams with limited resources. By integrating multiple specialized models within a single orchestrator, it showcases how modular design can balance performance and efficiency, offering a blueprint for building robust, offline-first AI applications that respect user privacy and data sovereignty.

Technical Details

• Orchestration: Built on LangGraph, the core chatbot.py manages state and routing across different operational lanes, allowing for complex multi-step reasoning and tool use.
• Model Configuration: Uses specific quantized models via Ollama, such as qwen2.5:3b-instruct for execution, qwen2.5:7b-instruct-q4_K_M for standard planning, and qwen2.5-coder:7b-instruct-q4_K_M for coding tasks, optimized for low VRAM usage.
• Interface & API: Provides a dual interface via a Tkinter-based desktop application (ui/) for direct interaction and a FastAPI server (api/) exposing endpoints via Server-Sent Events for programmatic access.
• Voice Pipeline: Implements an end-to-end offline voice system featuring wake-word detection, speech-to-text via faster-whisper, LLM processing, and text-to-speech using Piper, supporting mid-sentence barge-in.
• Memory & Tools: Features per-thread Retrieval-Augmented Generation (RAG) with a vector database and FIFO eviction policy, alongside a secure script executor for running uploaded .py files with human approval gates.

Industry Insight

Developers should consider modular, multi-model architectures for edge deployments, as separating planning and execution roles allows for better resource management and cost efficiency compared to monolithic large models. The emphasis on local-only telemetry and human-in-the-loop controls highlights a growing market demand for transparent, privacy-centric AI tools that give users explicit control over data handling and automated actions. Additionally, supporting both local inference and cloud fallbacks provides a flexible deployment strategy that accommodates varying hardware capabilities, ensuring broader accessibility for AI agents.

TL;DR

• Ved 是一个资源受限的本地 AI 代理，支持 Windows/Linux/macOS，默认完全离线运行，最低仅需 12GB VRAM/32GB RAM。
• 采用 LangGraph 编排器，提供三种模式：标准闲聊（3B CPU）、Turbo/Coder（8B/7B 规划器 + 3B 执行器），并支持通过 OpenRouter 回退到云端 API。
• 集成完整的离线语音管道（Wake-word -> Faster-Whisper -> LLM -> Piper TTS），支持句中打断，并具备每线程独立的 RAG 向量数据库索引。
• 包含 Tkinter 桌面 UI 和 FastAPI HTTP 服务器，支持人类在环（Human-in-the-loop）审批文件系统和代码执行操作，确保安全性。
• 项目结构清晰，模块化设计，包含本地遥测功能（可禁用），旨在为个人用户或小团队提供低门槛、高隐私的 AI 助手解决方案。

为什么值得看

对于希望在本地部署具备工具调用能力和语音交互的 AI 代理的开发者而言，Ved 提供了一个轻量级且可定制的参考实现。它展示了如何在有限硬件资源下，通过混合使用不同规模的模型（规划器与执行器分离）来平衡性能与成本，同时强调了数据隐私和本地优先的设计哲学。

技术解析

• 架构与编排：核心基于 LangGraph (chatbot.py)，采用多车道（Lanes）设计。standard 车道处理简单对话；turbo 和 coder 车道采用“规划器-执行器”（Planner-Executor）架构，其中规划器负责复杂推理和工具选择，执行器负责具体代码或文件操作。
• 模型配置与硬件要求：默认离线模式下，规划器使用 7B-8B 量化模型（如 Qwen2.5-7B），执行器使用 3B 模型（如 Qwen2.5-3B），需约 12GB VRAM。若使用云端 API（如 OpenRouter），可降低至 4GB VRAM/16GB RAM。
• 语音与 RAG 集成：语音管道完全离线，使用 Faster-Whisper 进行 ASR，Piper 进行 TTS，支持实时打断。RAG 系统基于每线程隔离的向量数据库，采用 FIFO 策略管理上下文窗口，支持上传 .py 等文件进行索引。
• 安全与控制：内置 command_processor.py 处理斜杠命令，并通过 api/runner.py 执行脚本。关键特性是“人类在环”机制，在执行文件系统写入或代码运行前需人工确认。此外，提供本地遥测记录活跃会话，可通过环境变量一键禁用。

行业启示

• 边缘 AI 的可行性验证：Ved 证明了通过模型分工（小模型执行+大模型规划）和量化技术，普通消费级硬件即可运行具备复杂工具调用能力的 AI 代理，降低了本地 AI 应用的门槛。
• 隐私优先的设计范式：默认离线、本地遥测、可选云端回退以及人类在环审批，体现了对数据主权和安全性的重视，这将是企业级和个人 AI 应用的重要差异化竞争点。
• 模块化代理架构趋势：将 UI、API、语音、RAG 和核心逻辑解耦，并支持多种后端（Ollama/OpenRouter），展示了构建灵活、可插拔 AI 代理系统的最佳实践，有助于开发者快速搭建定制化智能体。