Ace the AI Engineer Interview: LLM Fundamentals 通过 AI 工程师面试：LLM 基础

Data augmentation in NLP is split into rule-based (cheaper, controllable, best for cold-start) and generative (stronger generalization, requires powerful models, best for mature stages) Transformers outperform RNNs in training speed and long-term dependency modeling but suffer from O(T²) memory and compute complexity versus RNNs' O(T) Encoder models use MLM with bidirectional attention for representation tasks, while decoder models use CLM with causal masking for generative tasks Multi-head atte 数据增强分为规则式（同义词替换、格式转换、噪声注入）和生成式（改写、回译、文本生成），前者成本低可控性强，后者泛化能力更强 Transformer相比RNN在训练时具有并行计算优势，但内存和计算复杂度为O(T²)，RNN为O(T) Encoder模型使用MLM预训练和全注意力机制，Decoder模型使用CLM预训练和因果掩码自注意力 注意力机制通过QK点积计算权重，缩放因子√d_k防止梯度消失，多头注意力提升模型表达能力 数据增强存在过拟合和噪声引入风险，需根据项目阶段（冷启动vs成熟期）权衡选择策略