Policy Blindness
Cross-source consensus on Policy Blindness from 1 sources and 5 claims.
1 sources · 5 claims
How it works
Risks & contraindications
Comparisons
Evidence quality
Highlighted claims
- Policy Blindness refers to standard ZO methods perturbing all layers uniformly despite heterogeneous layer sensitivity. — Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
- Adam shows heterogeneous layer update patterns in OPT-6.7B, with larger updates in shallow and middle layers. — Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
- Low correlation between MeZO cumulative updates and Adam gradient norms indicates isotropic ZO exploration fails to recover intrinsic layer sensitivity. — Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
- MeZO cumulative updates are much more uniform than Adam's layer-wise update profile. — Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
- Dense uniform perturbation can inject noise through insensitive layers. — Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling