Fisher Information
Cross-source consensus on Fisher Information from 1 sources and 5 claims.
1 sources · 5 claims
How it works
Preparation
Risks & contraindications
Highlighted claims
- Fed-FSTQ uses token-level sensitivity as a rate-distortion signal for deciding what to transmit and at what precision. — FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
- The theoretical compression objective is Fisher-weighted rate-distortion. — FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
- The method uses squared embedding-gradient norms as token-level empirical Fisher proxies. — FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
- The method relies on diagonal Fisher approximations rather than full Fisher matrices, which improves tractability but ignores off-diagonal curvature. — FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
- Token sensitivity scores are smoothed with an exponential moving average using rho 0.9 by default. — FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices