Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion
The paper introduces **fact generation** for hyper-relational knowledge graphs (HKGs) as a more general task than traditional link prediction, where multiple or all components of a fact can be missing. To address this, the authors propose **KREPE**, a novel generative representation learning method that uses masked discrete diffusion to learn probability distributions over missing components, conditioned on local and global graph context. This model unifies link prediction and fact generation, a
Deep Analysis
Background
Current methods for inferring new knowledge in Hyper-relational Knowledge Graphs (HKGs) predominantly frame the problem as simple link prediction. This approach operates under the restrictive assumption that a fact is almost fully specified, with only a single missing element to be inferred. However, this does not reflect real-world scenarios where multiple or all components of a complex fact—such as entities, relations, and qualifiers—may be unknown or missing simultaneously. This gap between the technical approach and practical application motivates the need for a more flexible inference task.
Key Points
The core contribution is the formalization of a new task and a novel model to solve it.
- New Task: Fact Generation: The paper defines fact generation as the task of generating a valid hyper-relational fact from a partially observed or completely empty (masked) query. This generalizes link prediction, as it must handle an arbitrary number of missing components.
- Model: KREPE (Knowledge Representation via Parameterized Explanations): This is the first generative representation learning method designed for HKGs.
- Methodology: KREPE learns to model the probability distributions of missing components. It does so by conditioning on observed local fact components and the global structure of the entire HKG.
- Key Mechanism: The model uses masked discrete diffusion as its generative process. It captures intra-fact dependencies through contextual message passing and inter-fact correlations by aggregating stochastically sampled contexts from the graph.
- Unification and Performance: KREPE provides a single training framework that seamlessly handles both the traditional link prediction task and the new, more comprehensive fact generation task. It achieves state-of-the-art performance on standard HKG link prediction benchmarks and demonstrates superior ability to generate novel and correct facts, outperforming baselines based on large language models (LLMs).
Significance
This research is significant for advancing knowledge graph completion beyond the limitations of prior work.
- Addressing a Realistic Gap: By introducing fact generation, the work moves the field closer to handling real-world incompleteness, where knowledge inference often starts from very sparse information.
- Technical Innovation: KREPE represents a methodological shift from discriminative (link prediction) to generative modeling for HKGs. Its use of diffusion models to generate structured graph facts is a novel application.
- Practical Unified Framework: The ability to train one model for both fine-grained link prediction and full fact generation enhances efficiency and applicability. Demonstrating superiority over LLM-based baselines highlights the value of specialized, graph-aware architectures over general-purpose models for structured knowledge reasoning.
Disclaimer: The above content is generated by AI and is for reference only.