Releases

OpenAI introduced instruction-following models trained with human feedback.

Cohere introduced its Embed model family for semantic representation and retrieval.

Microsoft releases DeBERTa V3 with replaced-token detection and gradient-disentangled embedding sharing.

OpenAI introduced Codex, a descendant of GPT-3 that translates natural language into code.

DeepMind publishes AlphaFold 2's methodology and releases source code, expanding access to high-accuracy protein prediction.

Google introduces LaMDA as a conversational language model designed for open-ended dialogue.

OpenAI introduced CLIP, a neural network trained on image-text pairs for transferable visual recognition.

OpenAI introduced DALL-E, a model that generates images from text descriptions.

Facebook AI released M2M-100, a multilingual machine-translation model translating directly between 100 languages.

OpenAI published GPT-3, demonstrating few-shot learning at 175 billion parameters.

OpenAI released Jukebox, a neural network that generated music including rudimentary singing as raw audio.

Microsoft introduces Turing-NLG, a 17-billion-parameter generative language model and, at the time, the largest published language model.

OpenAI completed the staged GPT-2 release with the 1.5-billion-parameter model and accompanying research.

Hugging Face released DistilBERT, a smaller and faster distilled BERT model retaining most of the teacher model’s language-understanding performance.

OpenAI released the 774-million-parameter GPT-2 model and shared further analysis of misuse and detection.

OpenAI introduced MuseNet, a deep neural network for generating multi-instrument musical compositions.

Facebook open-sourced its Deep Learning Recommendation Model reference implementation.

Baidu released its first knowledge-enhanced language representation model.

OpenAI introduced GPT-2 and began a staged release process due to misuse concerns.

Google open-sources BERT, enabling bidirectional pre-training for language understanding and downstream NLP tasks.

OpenAI presented its five-agent Dota 2 system trained through large-scale self-play.

Baidu opened PaddlePaddle to the global developer community for deep learning, vision, speech and NLP.

Google releases TensorFlow as an open-source machine-learning framework, making internal deep-learning infrastructure broadly available.

IBM publicly demonstrated its DeepQA question-answering architecture through the Jeopardy! competition.