Ask HN: How would you harden AI changes to a 1M-line legacy SaaS before review?
An experiment demonstrated that agentic development can build a 13k-line MVP with equal test coverage in two weeks using a multi-agent workflow (planner, coder, reviewer) on a legacy C#/React codebase. The process utilized distinct agents for planning, implementation, and review, with a "stop and ask" mechanism for autonomous escalation, reducing the need for full-time human developer involvement. Dual-model review proved critical, as a second model (Codex) identified substantially more issues i
Analysis
TL;DR
- An experiment demonstrated that agentic development can build a 13k-line MVP with equal test coverage in two weeks using a multi-agent workflow (planner, coder, reviewer) on a legacy C#/React codebase.
- The process utilized distinct agents for planning, implementation, and review, with a "stop and ask" mechanism for autonomous escalation, reducing the need for full-time human developer involvement.
- Dual-model review proved critical, as a second model (Codex) identified substantially more issues in code generated by the primary model (Claude), highlighting the value of diverse agent perspectives.
- Key challenges remain regarding code robustness, the risk of shared incorrect assumptions between coder and reviewer agents, and the gap between prototype functionality and production-grade engineering standards.
- The author seeks strategic advice on maximizing pre-review confidence through independent testing, documentation, and prioritization to minimize the effort required for final engineering conversion.
Why It Matters
This case study provides a realistic benchmark for the current state of AI-driven software engineering, demonstrating that while agents can accelerate prototyping significantly, they do not yet fully replace human judgment for production readiness. It highlights the importance of multi-agent collaboration and cross-model validation in mitigating hallucinations and logical errors, offering a blueprint for organizations attempting to leverage AI for rapid MVP development under resource constraints.
Technical Details
- Workflow Architecture: A three-agent system consisting of a Planner (converting epics to Jira/MD stories), a Coder (implementing stories and tests), and a Reviewer (auditing PRs). The Coder could escalate unresolved issues back to the Planner.
- Tooling Stack: Primary implementation used Claude-based agents, with Codex employed as a secondary reviewer for high-risk pull requests. Development occurred in an isolated branch with a dedicated database schema.
- Code Metrics: The resulting MVP consisted of approximately 13,000 lines of functional code and 13,000 lines of tests, developed over a two-week period after a two-week setup phase.
- Validation Strategy: Daily manual end-to-end testing was conducted by the project lead, alongside automated refactoring and hardening runs. A dual-model review process was implemented to catch errors missed by the primary coding agent.
- Constraints: Token quotas limited the extensive use of the secondary reviewer, and human intervention was restricted to resolving specific escalations and occasional technical guidance.
Industry Insight
- Adopt Multi-Agent Diversity: To mitigate systemic errors, organizations should implement workflows where different AI models or specialized agents handle generation and verification, ensuring that the reviewer does not share the same blind spots as the coder.
- Prioritize Test Independence: Generating tests via a separate agent from the one writing the implementation is crucial for catching edge cases and logic errors, serving as a stronger quality gate than self-verification.
- Invest in Pre-Review Documentation: Structuring PRs with clear architectural guardrails, interaction details, and evidence of manual testing will significantly reduce the cognitive load on human engineers during the final production-readiness review.
Disclaimer: The above content is generated by AI and is for reference only.