Template:Differentiable computing

Differentiable computing

Concepts

Clustering
Regression
Backpropagation
Autoregression
Gradient descent
- SGD
- Overfitting
Hallucination
Adversary
Attention
Convolution
Loss functions
Batchnorm
Activation
- Softmax
- Sigmoid
- Rectifier
Regularization
Datasets
- Augmentation
Diffusion
Differentiable function
Differentiable programming
Information geometry
Statistical manifold
Automatic differentiation
Neuromorphic engineering
Pattern recognition
Tensor calculus
Computational learning theory
Inductive bias

Applications

Machine learning
- In-context learning
Artificial neural network
- Deep learning
Scientific computing
Artificial Intelligence
Language model
- Large language model

Hardware

IPU
TPU
VPU
Memristor
SpiNNaker

Software libraries

TensorFlow
PyTorch
Keras
Theano
JAX
Flux.jl
MindSpore

Implementations

Audio–visual	AlexNet WaveNet Human image synthesis HWR OCR Speech synthesis Speech recognition Facial recognition AlphaFold Text-to-image models DALL-E Midjourney Stable Diffusion Text-to-video models Sora VideoPoet Whisper
Verbal	Word2vec Seq2seq BERT Gemini LaMDA Bard NMT Project Debater IBM Watson IBM Watsonx Granite GPT-1 GPT-2 GPT-3 GPT-4 ChatGPT GPT-J Chinchilla AI PaLM BLOOM LLaMA PanGu-Σ
Decisional	AlphaGo AlphaZero Q-learning SARSA OpenAI Five Self-driving car MuZero Action selection Auto-GPT Robot control

People

Yoshua Bengio
Alex Graves
Ian Goodfellow
Stephen Grossberg
Demis Hassabis
Geoffrey Hinton
Yann LeCun
Fei-Fei Li
Andrew Ng
Jürgen Schmidhuber
David Silver
Ilya Sutskever

Organizations

Anthropic
EleutherAI
Google DeepMind
Hugging Face
OpenAI
Meta AI
Mila
MIT CSAIL
Huawei

Architectures

Neural Turing machine
Differentiable neural computer
Transformer
Recurrent neural network (RNN)
Long short-term memory (LSTM)
Gated recurrent unit (GRU)
Echo state network
Multilayer perceptron (MLP)
Convolutional neural network
Residual neural network
Mamba
Autoencoder
Variational autoencoder (VAE)
Generative adversarial network (GAN)
Graph neural network