Module mpt 

Module implementing the MPT (Multi-Purpose Transformer) model

References:

• MPT Model used by replit-code-v1_5-3b
• Configuration

The model uses grouped query attention and alibi positional embeddings.

Structs

Config

Model