Releases

Microsoft Research introduces DeBERTa, improving Transformer language models with disentangled attention and enhanced mask decoding.

OpenAI published GPT-3, demonstrating few-shot learning at 175 billion parameters.

Hugging Face launched hosted dataset repositories alongside its Datasets library, creating a shared catalog for reusable machine-learning data.

Hugging Face released a unified open-source library and catalog for loading, processing, streaming, and sharing datasets.

NVIDIA began shipping DGX A100, an integrated 5-petaflop AI system with eight A100 GPUs.

NVIDIA launched the A100 Tensor Core GPU, its first Ampere data-center accelerator for training and inference.

Hugging Face made model repositories a central Hub product for versioning, documenting, discovering, and reusing pretrained models.

Hugging Face expanded from an NLP library into a Git-based collaboration platform for sharing models and machine-learning artifacts.

OpenAI released Jukebox, a neural network that generated music including rudimentary singing as raw audio.

AWS and Facebook jointly released TorchServe, an open-source model-serving library for PyTorch.

Hugging Face introduced its high-performance Rust-backed Tokenizers library for production-scale text preprocessing.

Microsoft open-sources DeepSpeed with the ZeRO optimizer to reduce the compute and memory cost of large-model training.

Oracle launched a managed collaborative environment for building, training and deploying machine-learning models.

Microsoft introduces Turing-NLG, a 17-billion-parameter generative language model and, at the time, the largest published language model.

Databricks articulated the lakehouse architecture, combining data-lake openness with data-warehouse management and performance.

Foundry ML capabilities became available for operational machine learning.

Model Studio became available for developing and operationalizing machine-learning models.

Foundry added managed coding environments for data science and software work.

Baidu released its open NLP development library for PaddlePaddle.

CoreWeave established a managed, multi-tenant Kubernetes cloud running directly on bare metal for GPU-accelerated machine-learning and inference workloads.

Scale began supporting U.S. defense and national-security AI programs, extending its data infrastructure into government missions.

Hugging Face raised $15 million in a Series A led by Lux Capital to expand its open-source natural-language-processing platform.

AWS launched automated model building with full visibility into generated models and notebooks.

AWS launched a web-based integrated development environment for machine learning.

AWS introduced managed human-review workflows for machine-learning predictions.