DistilBERT
ModelRelease date and specs of Hugging Face's AI model
Distilled version of BERT created by Hugging Face to retain much of BERT’s capability with substantially fewer parameters and faster inference.
- Released
- Oct 2, 2019
- Context window
- —
- Modality
- text
- Open source
- Yes
- API available
- —
- Releases
- 1
- Parameters
- 66M
Releases
Oct 2, 2019DistilBERT releasedModel release
Hugging Face released DistilBERT, a smaller and faster distilled BERT model retaining most of the teacher model’s language-understanding performance.
DistilBERT FAQ
When was DistilBERT released?
DistilBERT was released by Hugging Face on Oct 2, 2019.
Is DistilBERT open source?
Yes. DistilBERT is listed as open source (open weights).

