RedPajama
Model FamilyTogether AIactive
Open model and dataset initiative recreating and extending high-quality pre-training data and language models.
- Apr 17, 2023
- —
- —
- Yes
- —
- 2
Releases
May 5, 2023Together releases RedPajama-INCITE 3B and 7B modelsOpen weights
Together released base, instruction-tuned, and chat RedPajama-INCITE language models at 3B and 7B parameters under Apache 2.0.
Apr 17, 2023Together releases RedPajama-Data-1TOpen weights
Together and collaborators released a reproducible, openly licensed 1.2-trillion-token pre-training dataset following the LLaMA data recipe.