Research
DeepMind paper: linear attention rivals transformers at 32K
A new state-space hybrid matches softmax attention quality with O(n) memory at long context.
Papers, findings, and ideas that will define what AI becomes next.
A new state-space hybrid matches softmax attention quality with O(n) memory at long context.
A comprehensive review of reinforcement-learned agents - from RLHF descendants to self-play loops.
Randomly masking retrieved passages during training improves grounded-answer accuracy.
Ternary-weight models maintain quality on GSM8K and MMLU at a fraction of memory and energy.