Gumbel Straight-Through Estimation
Cross-source consensus on Gumbel Straight-Through Estimation from 1 sources and 5 claims.
1 sources · 5 claims
How it works
Dosage & preparation
Comparisons
Evidence quality
Highlighted claims
- Temperature is annealed exponentially from 1.0 to 0.01 in the described training procedure. — Budget Constraints as Riemannian Manifolds
- The full convergence behavior with Adam, Gumbel noise, and temperature annealing is empirical rather than proven. — Budget Constraints as Riemannian Manifolds
- In Qwen3-8B ablations, Gumbel sample count was the most important hyperparameter. — Budget Constraints as Riemannian Manifolds
- More samples per step were favored over more steps in compute-matched comparisons. — Budget Constraints as Riemannian Manifolds
- RCO samples Gumbel noise, perturbs logits, and averages multiple samples per step to reduce variance. — Budget Constraints as Riemannian Manifolds