The papers that built modern AI
25 landmark papers in five reading paths, for undergrad and grad students who want to read the originals. Each paper card covers the problem, the method, what changed, and what is still open.
Deep learning foundations
The training toolkit every later paper assumes: how networks learn, why deep nets suddenly worked, and the tricks that keep training stable.
- 01Learning representations by back-propagating errorsBeginner
Backpropagation, the engine of all neural network training.
- 02ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)Beginner
GPUs plus deep CNNs won ImageNet by a wide margin and started the deep learning era.
- 03Dropout: A Simple Way to Prevent Neural Networks from OverfittingBeginner
The standard regulariser, still in most architectures.
- 04Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftIntermediate
Made very deep nets trainable; the idea behind the normalisation layers in Transformers.
- 05Adam: A Method for Stochastic OptimizationIntermediate
The default optimiser for almost every model trained today.
Transformers and language models
From sequence models to today’s large language models, in the order the ideas arrived.
- 06Sequence to Sequence Learning with Neural NetworksIntermediate
The encoder-decoder pattern that framed machine translation.
- 07Neural Machine Translation by Jointly Learning to Align and TranslateIntermediate
Introduced attention, the idea the Transformer is built on.
- 08Intermediate
- 09BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPaper cardIntermediate
Pre-train, then fine-tune: the recipe that took over NLP.
- 10Language Models are Few-Shot Learners (GPT-3)Intermediate
Scale alone unlocked in-context learning and prompting.
- 11Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsBeginner
Step-by-step reasoning, the seed of today’s reasoning models.
Computer vision and generative models
How models learned to see, then to create images, then to connect images with language.
- 12Deep Residual Learning for Image Recognition (ResNet)Intermediate
Skip connections let networks go hundreds of layers deep.
- 13Generative Adversarial NetsIntermediate
Two networks competing: the first convincing image generators.
- 14Denoising Diffusion Probabilistic ModelsAdvanced
The diffusion approach behind today’s image and video generators.
- 15An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale (ViT)Intermediate
Brought the Transformer to vision and unified the field’s architecture.
- 16Learning Transferable Visual Models From Natural Language Supervision (CLIP)Intermediate
Put images and text in one shared space, a key step toward multimodal models.
Reinforcement learning and robotics
Learning by trial and reward, from Atari to robots that follow language. Pairs with AI Robotics.
Humanoids, robotaxis, warehouse fleets and the AI that drives them, tracked daily.
- 17Human-level control through deep reinforcement learning (DQN)Intermediate
Deep nets plus RL learned Atari games straight from pixels.
- 18Mastering the game of Go with deep neural networks and tree search (AlphaGo)Intermediate
Combined learning and search to beat top human Go players.
- 19Proximal Policy Optimization Algorithms (PPO)Advanced
The workhorse RL algorithm, later used in RLHF for language models.
- 20Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real WorldIntermediate
Train in simulation, deploy on real robots: the sim-to-real idea.
- 21RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic ControlAdvanced
Web-trained vision-language models driving robot actions, a root of today’s robot foundation models.
Scaling, alignment and efficiency
Why bigger models got better, how they were made helpful, and how students can fine-tune them on one GPU.
- 22Scaling Laws for Neural Language ModelsIntermediate
Loss falls predictably as model size, data and compute grow.
- 23Training Compute-Optimal Large Language Models (Chinchilla)Intermediate
Corrected the recipe: train on far more data per parameter.
- 24Training language models to follow instructions with human feedback (InstructGPT)Intermediate
RLHF, the method that turned GPT-3 into a usable assistant.
- 25LoRA: Low-Rank Adaptation of Large Language ModelsIntermediate
Cheap fine-tuning, the technique students actually use in projects.
How to use this hub
Read each path top to bottom, since earlier papers set up later ones. Links marked ↗ open the original paper. Paper cards are researched and drafted by AIooze agents and checked against the original. Always cite the original paper, not this page.