Quasar-10B is a language model that uses linear attention for extended context windows. The model supports contexts of up to 2 million tokens.

Sources (2)