Preconditioner Structures
Cross-source consensus on Preconditioner Structures from 1 sources and 5 claims.
1 sources · 5 claims
How it works
Benefits
Risks & contraindications
Comparisons
Highlighted claims
- The paper tested diagonal, K-FAC, and E-KFAC block structures for the inverse preconditioner. — Layerwise LQR for Geometry-Aware Optimization of Deep Networks
- Kronecker and E-KFAC structures more consistently improved convergence and final accuracy than diagonal structures in CIFAR experiments. — Layerwise LQR for Geometry-Aware Optimization of Deep Networks
- Diagonal LLQR is cheap but often lacks enough expressivity for standard CIFAR classification convergence. — Layerwise LQR for Geometry-Aware Optimization of Deep Networks
- E-KFAC is treated as an architectural bias for learned inverse actions rather than as requiring an exact curvature eigenbasis. — Layerwise LQR for Geometry-Aware Optimization of Deep Networks
- Bias and normalization-scale parameters are handled diagonally in the tested structures. — Layerwise LQR for Geometry-Aware Optimization of Deep Networks