Releases

Facebook AI released M2M-100, a multilingual machine-translation model translating directly between 100 languages.

OpenAI published GPT-3, demonstrating few-shot learning at 175 billion parameters.

OpenAI released Jukebox, a neural network that generated music including rudimentary singing as raw audio.

Microsoft introduces Turing-NLG, a 17-billion-parameter generative language model and, at the time, the largest published language model.

OpenAI completed the staged GPT-2 release with the 1.5-billion-parameter model and accompanying research.

Hugging Face released DistilBERT, a smaller and faster distilled BERT model retaining most of the teacher model’s language-understanding performance.

OpenAI released the 774-million-parameter GPT-2 model and shared further analysis of misuse and detection.

OpenAI introduced MuseNet, a deep neural network for generating multi-instrument musical compositions.

Facebook open-sourced its Deep Learning Recommendation Model reference implementation.

Baidu released its first knowledge-enhanced language representation model.

OpenAI introduced GPT-2 and began a staged release process due to misuse concerns.

Google open-sources BERT, enabling bidirectional pre-training for language understanding and downstream NLP tasks.

OpenAI presented its five-agent Dota 2 system trained through large-scale self-play.

Baidu opened PaddlePaddle to the global developer community for deep learning, vision, speech and NLP.

Google releases TensorFlow as an open-source machine-learning framework, making internal deep-learning infrastructure broadly available.

IBM publicly demonstrated its DeepQA question-answering architecture through the Jeopardy! competition.