GraphLLMRec

GNN + LLM Hybrid Recommendation with Grokking Detection

GraphLLMRec combines graph neural networks (GNN) with large language models for recommendation, exploiting the complementary strengths of structural user-item interaction modeling (GNN) and semantic content understanding (LLM). The GNN component uses LightGCN and GraphSAGE to learn collaborative filtering signals from the interaction graph, while Qwen2.5-3B provides content-based understanding of item descriptions and user reviews.

A novel contribution is the grokking detection mechanism that monitors for sudden generalization improvements during training -- the phenomenon where models abruptly transition from memorization to generalization long after reaching near-zero training loss. GraphLLMRec automatically detects grokking events and adjusts the fusion weight between GNN and LLM components based on which modality is grokking, ensuring that the late-generalizing component is given appropriate influence.

The hybrid architecture uses a learned gating mechanism to fuse GNN embeddings and LLM representations at the item level, with separate gates for different recommendation contexts (cold-start vs. warm, popular vs. long-tail). This context-aware fusion outperforms static weighted combination by 5-8% in NDCG@20 across benchmarks.

Built on PyTorch and DGL (Deep Graph Library), the system supports efficient mini-batch training on large-scale graphs with neighbor sampling, enabling training on million-scale interaction datasets within practical time budgets.

• GNN (LightGCN/GraphSAGE) + LLM (Qwen2.5-3B) hybrid recommendation architecture
• Grokking detection for automatic fusion weight adjustment during training
• Context-aware gating: 5-8% NDCG@20 improvement over static fusion
• DGL-based efficient mini-batch training on million-scale interaction graphs
• Separate handling for cold-start vs. warm and popular vs. long-tail items