J-Miner: Recovering Executable Decision Knowledge from Language-Model Classifiers
J-Miner mines internal decision knowledge from fine-tuned language model classifiers and encodes it into explicit, executable decision rules The method aggregates vocabulary-aligned internal signals across layers and token positions to discover text-level named concepts J-Miner rules reproduce up to 98.3% of source-classifier decisions, outperforming rules learned from input words by 6.0–29.5 percentage points in behavioral fidelity The extracted decision knowledge transfers to lightweight stude
Analysis
TL;DR
- J-Miner mines internal decision knowledge from fine-tuned language model classifiers and encodes it into explicit, executable decision rules
- The method aggregates vocabulary-aligned internal signals across layers and token positions to discover text-level named concepts
- J-Miner rules reproduce up to 98.3% of source-classifier decisions, outperforming rules learned from input words by 6.0–29.5 percentage points in behavioral fidelity
- The extracted decision knowledge transfers to lightweight student models using ~1/24 the parameters while retaining 99.8% of the source classifier's mean task accuracy
- This demonstrates that task-specific decision knowledge can be faithfully represented in an explicit, inspectable, and reusable form beyond the original classifier
Why It Matters
This work addresses a critical gap in AI interpretability: while fine-tuned language models achieve strong classification performance, their decision-making process remains opaque. J-Miner provides a practical pathway to extract and externalize that implicit knowledge, enabling inspection, validation, and reuse—key requirements for deploying trustworthy AI systems in high-stakes domains.
Technical Details
- J-Miner operates by aggregating vocabulary-aligned internal activation signals across transformer layers and token positions to identify text-level named concepts that the classifier implicitly relies on
- Executable decision rules are learned directly from the classifier's own predictions over these mined concepts, distilling distributed internal readouts into a classifier-level knowledge representation
- The named concepts discovered reflect internal semantic evidence associated with task decisions, and the learned rules consolidate these signals into inspectable decision structures
- Lightweight standalone student models (~1/24 the parameters of source classifiers) can reconstruct and execute the mined representation from raw text, retaining 99.8% of the source classifiers' mean task accuracy
- Evaluated across multiple classification tasks, J-Miner rules achieve 6.0–29.5 percentage points higher behavioral fidelity compared to equally compact rules learned solely from input words
Industry Insight
- J-Miner's approach offers a practical interpretability tool for organizations deploying fine-tuned LLMs, enabling them to audit and validate model decisions without requiring access to the full model weights
- The successful transfer to lightweight students suggests a viable model compression pathway: extract interpretable rules from large classifiers and deploy compact, transparent alternatives in production
- As regulatory scrutiny of AI decision-making increases, methods like J-Miner that produce executable, inspectable decision knowledge will become increasingly valuable for compliance and trust-building in AI systems
Disclaimer: The above content is generated by AI and is for reference only.