Releases

Microsoft Research introduces DeBERTa, improving Transformer language models with disentangled attention and enhanced mask decoding.

OpenAI published GPT-3, demonstrating few-shot learning at 175 billion parameters.

Hugging Face launched hosted dataset repositories alongside its Datasets library, creating a shared catalog for reusable machine-learning data.

Hugging Face released a unified open-source library and catalog for loading, processing, streaming, and sharing datasets.

NVIDIA began shipping DGX A100, an integrated 5-petaflop AI system with eight A100 GPUs.

NVIDIA launched the A100 Tensor Core GPU, its first Ampere data-center accelerator for training and inference.

Hugging Face made model repositories a central Hub product for versioning, documenting, discovering, and reusing pretrained models.

Hugging Face expanded from an NLP library into a Git-based collaboration platform for sharing models and machine-learning artifacts.

OpenAI released Jukebox, a neural network that generated music including rudimentary singing as raw audio.

AWS and Facebook jointly released TorchServe, an open-source model-serving library for PyTorch.

Hugging Face introduced its high-performance Rust-backed Tokenizers library for production-scale text preprocessing.

Microsoft open-sources DeepSpeed with the ZeRO optimizer to reduce the compute and memory cost of large-model training.

Oracle launched a managed collaborative environment for building, training and deploying machine-learning models.

Microsoft introduces Turing-NLG, a 17-billion-parameter generative language model and, at the time, the largest published language model.

Databricks articulated the lakehouse architecture, combining data-lake openness with data-warehouse management and performance.

Foundry ML capabilities became available for operational machine learning.

Model Studio became available for developing and operationalizing machine-learning models.

Foundry added managed coding environments for data science and software work.

Baidu released its open NLP development library for PaddlePaddle.

CoreWeave established a managed, multi-tenant Kubernetes cloud running directly on bare metal for GPU-accelerated machine-learning and inference workloads.

Scale began supporting U.S. defense and national-security AI programs, extending its data infrastructure into government missions.

Hugging Face raised $15 million in a Series A led by Lux Capital to expand its open-source natural-language-processing platform.

AWS launched automated model building with full visibility into generated models and notebooks.

AWS launched a web-based integrated development environment for machine learning.

AWS introduced managed human-review workflows for machine-learning predictions.

AWS introduced an intelligent enterprise-search service using machine learning and natural-language queries.

AWS introduced ML-powered code review and performance recommendations.

AWS made Inferentia-powered EC2 instances generally available for machine-learning inference.

AWS introduced a generative-AI learning service and musical keyboard.

CoreWeave launched its specialized cloud business around on-demand NVIDIA GPU infrastructure for compute-intensive workloads.

After the cryptocurrency downturn, the company redirected its GPU fleet toward accelerated cloud workloads and changed its name to CoreWeave, Inc.

OpenAI completed the staged GPT-2 release with the 1.5-billion-parameter model and accompanying research.

Microsoft organizes Project Turing to develop large language models shared across Bing, Office, Dynamics and Azure.

Google researchers publish the T5 approach and Colossal Clean Crawled Corpus, framing diverse NLP tasks in one text-to-text format.

Databricks raised $400 million in Series F financing at a $6.2 billion valuation.

NVIDIA expanded EGX with certified servers, software and ecosystem support for AI, IoT and 5G at the edge.

Databricks introduced the MLflow Model Registry for centralized model versioning and lifecycle management.

Facebook AI released Detectron2, a modular object-detection and segmentation library.

Hugging Face released DistilBERT, a smaller and faster distilled BERT model retaining most of the teacher model’s language-understanding performance.

Hugging Face renamed pytorch-transformers to Transformers and expanded the library’s framework and architecture support.

Facebook agreed to acquire CTRL-labs, bringing neural-interface research into Reality Labs.

OpenAI released the 774-million-parameter GPT-2 model and shared further analysis of misuse and detection.

Scale raised $100 million in Series C funding led by Founders Fund at a valuation above $1 billion.

Baidu released an open OCR toolkit built on PaddlePaddle.

Baidu published a continual pre-training framework for incremental multi-task learning.

Microsoft invests $1 billion in OpenAI and becomes its exclusive cloud partner for new Azure AI supercomputing technologies.

Microsoft invested $1 billion in OpenAI and the companies formed an exclusive cloud-computing partnership.

The completed Red Hat acquisition established the hybrid-cloud and open-source platform underpinning IBM's later AI portfolio.

IBM completed its acquisition of Red Hat to anchor an open hybrid-cloud platform for enterprise software and AI.

Facebook AI introduced wav2vec, a self-supervised approach to speech representation learning from raw audio.

Facebook AI released AI Habitat, a simulation platform for embodied AI research.

Google releases MediaPipe as a cross-platform framework for building perception and multimodal ML pipelines.

NVIDIA introduced EGX to bring real-time AI computing to edge and industrial environments.

Microsoft open-sources InterpretML for training interpretable models and explaining black-box predictions.

OpenAI introduced MuseNet, a deep neural network for generating multi-instrument musical compositions.

OpenAI Five defeated the reigning Dota 2 world champion team OG in a live best-of-three match.

FAIR researchers published a transformer language model that learned biological properties from protein sequences.

Facebook open-sourced its Deep Learning Recommendation Model reference implementation.

AWS released optimized Docker images for deep-learning training and inference.

Baidu released its first knowledge-enhanced language representation model.

Baidu introduced knowledge-enhanced pre-training through ERNIE.

OpenAI created a capped-profit entity, OpenAI LP, to scale investment in compute and talent while keeping the nonprofit mission in control.

MLflow 1.0 marked the first stable major release of the open-source ML lifecycle platform.

OpenAI introduced GPT-2 and began a staged release process due to misuse concerns.

Databricks raised $250 million led by Andreessen Horowitz, valuing the company at $2.75 billion.

Pipeline Builder became available for visual data-transformation authoring in Foundry.

Apollo's continuous-delivery capabilities became the deployment layer for Palantir software.

Workshop became available for building operational applications on the Ontology.

Palantir launched Apollo to continuously deliver and manage software across heterogeneous environments.

Aidan Gomez, Nick Frosst and Ivan Zhang founded Cohere to build broadly usable natural-language AI, later focusing on enterprise deployment.

Scale introduced a managed workspace for configuring annotation projects, instructions, quality controls, and review workflows.

Gradio launched as an open-source Python framework for rapidly building browser interfaces around machine-learning models.

Microsoft open-sources ONNX Runtime as a cross-platform high-performance inference engine.

DeepMind's first AlphaFold system achieves top performance in the CASP13 protein-structure prediction assessment.

Google Research releases JAX for composable automatic differentiation and accelerator-backed numerical computing.

AWS launched attachable fractional GPU acceleration for EC2 and SageMaker inference workloads.

AWS announced its first custom machine-learning inference chip.

AWS launched a reinforcement-learning education platform built around an autonomous scale race car.

AWS introduced document AI for extracting text, handwriting, forms and tables.

AWS introduced a managed forecasting service based on technology used in Amazon's retail operations.

AWS introduced a managed service for building individualized recommendations using technology derived from Amazon.com.

AWS introduced a managed data-labeling service combining human annotation with active learning.

AWS launched a HIPAA-eligible NLP service for extracting medical entities and relationships from text.

Google open-sources BERT, enabling bidirectional pre-training for language understanding and downstream NLP tasks.

Oracle launched a multi-skill enterprise digital assistant using natural-language understanding and machine learning.

Hugging Face released an open-source PyTorch implementation and pretrained checkpoints for BERT, the project that evolved into the Transformers library.

NVIDIA launched RAPIDS as an open-source GPU-accelerated data science and machine-learning suite.

NVIDIA launched the T4 GPU and TensorRT 5 as a data-center inference platform.

Scale expanded its annotation platform around the image, video, lidar, and sensor-fusion requirements of autonomous-vehicle developers.

Scale raised $18 million in Series B funding led by Index Ventures with participation from Accel, Y Combinator, and other investors.

Baidu announced its first Kunlun AI accelerator for cloud and edge workloads.

Baidu introduced its in-house Kunlun AI computing architecture.

Baidu launched a full-featured machine-learning platform for algorithm developers.

Apollo 3.0 added production support for autonomous minibuses, delivery vehicles and valet parking.

Baidu launched a PaddlePaddle development and education platform for model building.

Baidu Brain 3.0 added multimodal semantic understanding and more than 110 AI capabilities.

OpenAI presented its five-agent Dota 2 system trained through large-scale self-play.

IBM Research demonstrated an AI system that generated evidence-backed arguments and debated a human expert.

OpenAI published the original GPT approach: generative pre-training followed by discriminative fine-tuning.

Databricks introduced MLflow as an open-source platform for managing the machine-learning lifecycle.