Expand description
StableLM model implementation.
StableLM is a family of language models trained by Stability AI. This implementation supports the StableLM architecture.
Key characteristics:
- Grouped query attention (GQA)
- Layer normalization
- Rotary positional embeddings (RoPE)
- Support for different model sizes (3B, 7B)
References:
- 🤗 Model Card