NVIDIA Releases Personal AI Router (PAIR): An Open Source Virtual Inference Router that Distributes Local AI Requests Across RTX, DGX Spark, and Mac Nodes NVIDIA发布个人AI路由器(PAIR)：开源虚拟推理路由器，跨RTX、DGX Spark和Mac节点分配本地AI请求

TL;DR

• NVIDIA released PAIR (Personal AI Router) as an open-source virtual inference router (Apache 2.0, public beta v0.1.1) that distributes multi-agent inference requests across multiple local machines on a home network
• PAIR does not introduce a new cluster API; it proxies existing Ollama and LM Studio endpoints, meaning agent harnesses require zero code changes to benefit from distributed scheduling
• Node eligibility is determined by five signals: online status, enabled engine, exact model presence, current job load, and GPU utilization; each request is assigned to a single node with no VRAM pooling or model sharding
• In NVIDIA's demo, a five-subagent Hermes Desktop workload on Qwen 3.6 35B A3B dropped from 18 minutes on a single RTX Spark laptop to 8 minutes 48 seconds across a three-device cluster (RTX Spark + DGX Spark + RTX 5090)
• Supported hardware includes GeForce RTX 20 Series and newer, RTX PRO Turing+, DGX Spark, and Apple M4+; cross-OS pairing (Windows, Linux, macOS) on x64 and arm64 is supported, with Windows on ARM marked experimental

Why It Matters

PAIR addresses a critical bottleneck in local multi-agent inference: when a lead agent spawns dozens of subagent calls, a single machine becomes a serialization point while other capable devices on the same network sit idle. By providing a drop-in routing layer that works with existing Ollama and LM Studio deployments, PAIR lets AI practitioners and researchers scale out local inference workloads across heterogeneous consumer and workstation hardware without rewriting agent code or managing a custom inference cluster.

Technical Details

• Architecture: PAIR is a virtual inference router, not a new inference engine. It sits between agent harnesses and local Ollama/LM Studio instances, proxying their default ports and exposing OpenAI-compatible endpoints. The agent decides what to request; PAIR decides where it runs.
• Discovery & Security: Nodes are discovered via mDNS (with manual IP fallback). Pairing uses a six-digit PIN exchanged between machines, and all node-to-node traffic is secured with mTLS using self-generated certificates. PAIR can also install engines and initiate model downloads on paired systems.
• Scheduling Policy: A request is eligible only when the target node has the required engine enabled and the exact model present. The scheduler weighs five signals—online readiness, engine enabled, model presence, job load, and GPU utilization. Routing is workload-level: each request stays on one node for its lifetime, with no VRAM pooling, GPU merging, or request sharding.
• Hardware & Platform Support: GeForce RTX 20 Series+, RTX PRO (Turing+), DGX Spark, Apple M4+. Cross-OS pairing across Windows, Linux, and macOS on x64 and arm64. Minimum 8 GB RAM, recommended 20 GB disk. Windows on ARM is experimental; other Linux distros build from source.
• Demo Performance: Five-subagent Hermes Desktop workload over a synthetic household inbox using Qwen 3.6 35B A3B on Ollama. Single RTX Spark laptop: 18 min average. Three-device PAIR cluster (RTX Spark + DGX Spark + RTX 5090): 8 min 48 s average. NVIDIA notes these are unofficial demo numbers.

Industry Insight

• The no-API-change design is a strategic move that lowers adoption friction dramatically; existing agent frameworks (Hermes, LangChain, etc.) can leverage distributed local inference immediately, which could accelerate the proliferation of multi-agent workflows on consumer hardware.
• The current single scheduling policy—blind to VRAM capacity, GPU class, and model warmness—suggests PAIR is in an early stage; expect future iterations to introduce cost-aware routing, model preloading heuristics, and possibly heterogeneous GPU class weighting to maximize throughput across mixed clusters.
• PAIR's existence signals NVIDIA's push to own the local inference orchestration layer alongside its hardware, complementing Ollama/LM Studio ecosystems rather than competing with them, which could position PAIR as a de facto standard for home-lab and edge multi-agent deployments.

TL;DR

• NVIDIA发布开源虚拟推理路由器PAIR（v0.1.1），解决多智能体工作流中本地推理请求竞争执行槽的瓶颈问题
• PAIR通过mDNS自动发现局域网内兼容设备，使用六位数PIN配对和mTLS加密建立安全通信
• 采用代理现有Ollama/LM Studio接口的设计，无需修改现有智能体框架即可接入
• 调度器基于节点在线状态、引擎启用、模型存在、任务负载和GPU利用率五个信号进行路由决策
• 演示中五子智能体工作流从单设备18分钟缩短至三设备集群8分48秒，性能提升约2倍

为什么值得看

PAIR填补了本地多智能体工作流分布式推理的基础设施空白，为个人/家庭AI部署提供了实用的横向扩展方案。其零代码修改的代理设计大幅降低了集成门槛，开源Apache 2.0许可为社区定制和二次开发提供了空间。

技术解析

PAIR不引入新的集群API，而是透明代理Ollama和LM Studio的默认端口，同时暴露OpenAI兼容端点。智能体框架无需任何改动，PAIR接管路由决策，现有引擎继续执行模型推理。

设备发现通过mDNS实现，支持手动IP添加作为备选。配对流程采用六位数PIN验证机制，未配对节点间通信被完全阻断，配对后使用生成的证书进行mTLS加密传输。

调度器将每个请求分配给单一节点，边界明确：不池化VRAM、不合并GPU、不分片模型。节点可用性取决于五个信号：在线状态、引擎启用、精确模型存在、当前任务负载和GPU利用率。

支持硬件包括GeForce RTX 20系列及以上、RTX PRO工作站GPU、DGX Spark和Apple M4及以上芯片，跨Windows/Linux/macOS运行（x64和arm64），最低配置要求8GB RAM和20GB磁盘空间。

行业启示

多智能体工作流正在重塑本地推理需求模式，从单请求串行转向大量并发独立请求。PAIR的出现标志着个人AI基础设施从"单设备单机"向"多设备协同"的架构演进，为边缘AI部署提供了新的扩展路径。

开源路由器方案降低了分布式推理的集成成本，但当前版本调度策略较为单一，对显存容量、GPU型号和模型预热状态不敏感，后续迭代可能向更精细的资源感知调度发展。