An Organizational Second Brain: Building an AI That Learns From Experts
Meta built an AI agent that serves as a secondary domain expert, preserving institutional knowledge in compliance and other specialist areas without requiring model retraining The system separates knowledge (what the agent knows) from reasoning (how it thinks) using a structured, auditable file-based architecture with 200+ organized files A self-improvement loop compiles expert feedback into verified, regression-tested updates, turning one-off corrections into permanent organizational memory The
Analysis
TL;DR
- Meta built an AI agent that serves as a secondary domain expert, preserving institutional knowledge in compliance and other specialist areas without requiring model retraining
- The system separates knowledge (what the agent knows) from reasoning (how it thinks) using a structured, auditable file-based architecture with 200+ organized files
- A self-improvement loop compiles expert feedback into verified, regression-tested updates, turning one-off corrections into permanent organizational memory
- The architecture includes position files, taxonomy/vocabulary files, routing indexes, and gateway files, each with explicit dependency graphs in YAML frontmatter
- The pattern generalizes beyond compliance to any enterprise domain governed by retrievable text, including finance, security, and engineering
Why It Matters
This represents a significant shift from RAG-based approaches that re-derive reasoning at inference time to a pre-structured knowledge system that makes implicit expert reasoning explicit and machine-readable. For AI practitioners, it demonstrates how to build agents that maintain citation fidelity and institutional consistency in high-stakes domains where off-the-shelf LLMs fall short due to lack of organizational context.
Technical Details
- Knowledge Architecture: 200+ files organized into a strict taxonomy—position files capture authoritative stances with constraints and routing implications; taxonomy/vocabulary files serve as single-source glossaries; routing indexes map inputs deterministically without relying solely on embedding similarity; gateway files define threshold tests before entering analytical domains
- Dependency Graph: Every file declares
depends_onandreferenced_byin YAML frontmatter, forming a bidirectional dependency graph that enables safe automated editing and change propagation - Self-Improvement Loop: Expert feedback is compiled into verified, regression-tested updates to knowledge files without retraining the underlying LLM, creating compounding institutional memory
- Four-Layer System: Knowledge system (organizational second brain), reasoning layer (mirrors expert thinking), evaluation framework (gates all changes), and improvement loop (feeds back into knowledge and reasoning)—each layer is interdependent
- Pre-Extraction Approach: A long-running offline process distills source documents into structured, machine-readable knowledge files ahead of time, avoiding slow and error-prone on-the-fly re-derivation from raw sources
Industry Insight
- The separation of knowledge from reasoning is a practical pattern for enterprise AI deployment—organizations should invest in structured knowledge curation rather than relying purely on retrieval-augmented generation for specialist domains
- Karpathy's LLM Wiki and Google's Open Knowledge Format signal industry convergence on file-based knowledge structures; building on these standards rather than proprietary approaches will improve cross-agent interoperability
- The regression-testing gate on knowledge updates is critical for production reliability—any enterprise AI system that modifies its own knowledge base must have automated evaluation frameworks to prevent drift and maintain citation fidelity
Disclaimer: The above content is generated by AI and is for reference only.