Unicorn, pelican, Middle-earth: OpenAI co-founder Karpathy is looking for the next AI vibe test
Andrej Karpathy prompted Claude Opus 5 to convert a single paragraph from "Lord of the Rings" into a fully functional 3D browser scene, producing 5,500 lines of Three.js code in approximately two hours The entire experiment cost roughly $10 using a one-million-token budget, demonstrating the feasibility of on-demand, low-cost procedural world generation The model autonomously placed and animated objects but introduced errors due to its inability to directly watch its own video output, relying in
Analysis
TL;DR
- Andrej Karpathy prompted Claude Opus 5 to convert a single paragraph from "Lord of the Rings" into a fully functional 3D browser scene, producing 5,500 lines of Three.js code in approximately two hours
- The entire experiment cost roughly $10 using a one-million-token budget, demonstrating the feasibility of on-demand, low-cost procedural world generation
- The model autonomously placed and animated objects but introduced errors due to its inability to directly watch its own video output, relying instead on static screenshots for self-evaluation
- Karpathy positions this as a "vibe check" rather than a rigorous benchmark, arguing that established tests like the pelican SVG challenge are approaching saturation
- The work signals emerging potential for cheap, user-customizable game worlds where individuals could be inserted as side characters or protagonists without manual development
Why It Matters
This experiment illustrates how frontier language models are transitioning from text generation to complex, multi-modal creative production at near-zero marginal cost. For AI practitioners and game developers, it demonstrates that high-quality 3D world generation is becoming accessible without traditional engineering pipelines, opening new pathways for procedural content creation and interactive entertainment.
Technical Details
- Claude Opus 5 generated 5,500 lines of Three.js code from a single narrative paragraph, running autonomously for approximately two hours on a one-million-token budget costing roughly $10
- The model handled spatial reasoning, object placement, and animation sequencing independently, though it relied on Eleven Labs for audio generation rather than producing it natively
- A key architectural limitation emerged: the model could not directly observe its own video output and instead depended on screenshot-based feedback loops, which introduced placement and animation errors
- Karpathy published the full source code publicly, enabling community playback, modification, and extension of the generated 3D scene
- The experiment follows a lineage of creative benchmarks including Microsoft's GPT-4 unicorn-in-TikZ test and Simon Willison's pelican-on-a-bicycle SVG challenge, both designed to assess spatial reasoning through novel, non-training-data prompts
Industry Insight
- The $10 cost for a functional 3D world suggests that procedural game development and interactive storytelling could see dramatic cost reductions, potentially enabling indie creators and platforms to offer on-demand custom environments at scale
- The screenshot-based self-evaluation bottleneck highlights a critical gap in current multimodal capabilities; models that can directly observe and iteratively refine their visual output will gain a significant advantage in creative and engineering tasks
- As vibe checks replace traditional benchmarks, the industry should develop new evaluation frameworks that measure not just correctness but creative fidelity, spatial coherence, and functional interactivity in generative outputs
Disclaimer: The above content is generated by AI and is for reference only.