KnowledgeForge: mining gold from the ITSM ticket graveyard
KnowledgeForge is an AI-powered system that mines resolved ITSM incident tickets to automatically generate knowledge base articles and root cause analysis documents, closing the gap between ticket resolution and knowledge capture. The system operates as a closed-loop pipeline: generation creates drafts from clustered tickets using RAG, while curation classifies, deduplicates, scores, and rewrites existing articles. Built on AWS infrastructure, it uses Amazon Bedrock (Claude Sonnet 4.5) for conte
Analysis
TL;DR
- KnowledgeForge is an AI-powered system that mines resolved ITSM incident tickets to automatically generate knowledge base articles and root cause analysis documents, closing the gap between ticket resolution and knowledge capture.
- The system operates as a closed-loop pipeline: generation creates drafts from clustered tickets using RAG, while curation classifies, deduplicates, scores, and rewrites existing articles.
- Built on AWS infrastructure, it uses Amazon Bedrock (Claude Sonnet 4.5) for content generation, Amazon S3 Vectors for semantic deduplication and grounding, and AWS Step Functions for orchestration.
- A human knowledge manager reviews and approves all output before it goes live, ensuring human oversight in the loop.
- The system reuses embedded vectors from curated articles as grounding context for future generation runs, continuously improving relevance and reducing duplication.
Why It Matters
KnowledgeForge addresses a critical pain point in enterprise IT operations: the vast majority of resolved incident knowledge is never captured in usable form, while existing knowledge bases grow messy and unreliable. For AI practitioners, it demonstrates a production-ready pattern for combining RAG, vector-based deduplication, and generative AI in a closed-loop workflow with human oversight—highly relevant for any organization managing large-scale document processing or knowledge management pipelines.
Technical Details
- Architecture: Two interconnected subsystems—Generation (ECS/Fargate) and Curation (Step Functions/Lambda)—operating in a closed loop where curation-embedded vectors feed back into generation as grounding context.
- Model: Anthropic Claude Sonnet 4.5 via Amazon Bedrock for both article generation and content improvement; Amazon Titan Text Embeddings V2 for vector embeddings stored in Amazon S3 Vectors.
- Generation Pipeline: Resolved tickets are clustered by theme and stored as JSON in S3. An ECS/Fargate container polls SQS, retrieves the 5 most similar existing articles from S3 Vectors for RAG grounding, then generates two structured documents per theme: a knowledge base article (title, summary, symptoms, root cause, resolution, prevention) and a root cause analysis document (executive summary, five-why analysis, corrective/preventive actions, cause code).
- Curation Pipeline: Every article (new and existing) passes through four Lambda-driven steps in Step Functions: type classification, duplicate detection via vector similarity, quality scoring, and content rewriting for weak articles.
- Infrastructure: AWS CDK-deployed stack including S3 buckets, S3 Vectors indexes, Step Functions state machines, Lambda functions, ECS/Fargate services, DynamoDB tables, SQS queues, Bedrock guardrails, and KMS keys. Output is sent to ServiceNow for human review, with approval decisions written back to DynamoDB.
Industry Insight
- The closed-loop design—where curated article embeddings continuously ground future generation—is a powerful pattern for any knowledge-intensive domain; organizations should design their AI systems to feed output back as input context rather than treating generation as a one-shot process.
- The hybrid compute strategy (long-running ECS/Fargate for generation bursts, serverless Lambda for curation workflows) demonstrates how to match workload characteristics to the right AWS compute model, a blueprint for building scalable generative AI pipelines.
- Human-in-the-loop review at the final stage, rather than at every step, balances automation speed with quality assurance—organizations should consider placing human review at the approval gate rather than trying to automate 100% of the pipeline.
Disclaimer: The above content is generated by AI and is for reference only.