Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks
Paper • 2603.11487 • Published • 2
Tel Aviv University
university
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks
ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA
YuvMil
submitted a
paper to Daily Papers 23 days ago
AviadDahan
submitted a
paper to Daily Papers 24 days ago
mega
authored 3
papers 25 days ago
Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models
Paper • 2602.02467 • Published
From Directions to Regions: Decomposing Activations in Language Models via Local Geometry
Paper • 2602.02464 • Published • 3
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
Paper • 2603.09906 • Published • 75
mega
authored a
paper 2 months ago
ItamarZ
submitted a
paper to Daily Papers 2 months ago
mega
authored a
paper 6 months ago
mega
authored 12
papers 7 months ago
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
Paper • 2303.09435 • Published
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion
Paper • 1902.10526 • Published
Injecting Numerical Reasoning Skills into Language Models
Paper • 2004.04487 • Published
LM vs LM: Detecting Factual Errors via Cross Examination
Paper • 2305.13281 • Published
Transformer Feed-Forward Layers Are Key-Value Memories
Paper • 2012.14913 • Published
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Paper • 2101.02235 • Published
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models
Paper • 2104.06129 • Published
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Paper • 2402.00559 • Published • 3
SCROLLS: Standardized CompaRison Over Long Language Sequences
Paper • 2201.03533 • Published • 1
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
Paper • 2203.14680 • Published
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
Paper • 2402.12865 • Published • 1
The Hidden Space of Transformer Language Adapters
Paper • 2402.13137 • Published