The useful history is not a leaderboard of model names. It is the sequence of abstractions that survived: shared representations, post-training, tool use, action-conditioned prediction, explicit evaluation, and hardware-aware execution.
This is a selective timeline, not an exhaustive catalog. I chose projects that changed how systems are built across six connected lanes. Dates refer to first papers or public releases, and “2026” means through July 25. Product rankings and “best model” claims are intentionally excluded because they age faster than the engineering ideas underneath them.
2021: one sequence interface for many problems
Three ideas set the base layer. CLIP learned a reusable image–text space from natural-language supervision; Codex treated source code as another language-model distribution; and LoRA showed that large frozen models could be adapted through small trainable low-rank updates. The shared pattern was more important than any one model: pretrain a broad representation, then specialize it without rebuilding the entire stack.
Decision-making moved toward the same interface. Decision Transformer cast offline RL as return-conditioned sequence modeling, while CLIPort combined semantic language–vision features with spatial transport structure for manipulation. Tokenization did not eliminate control, geometry, or reward; it made them interoperable with the scaling machinery of transformers.
2022: behavior became a post-training problem
InstructGPT made supervised instruction data, preference labels, a reward model, and RLHF a recognizable production pipeline. The lesson was that pretraining capability and user-facing behavior are different optimization stages. At the same time, Chinchilla reframed scale as a compute–data allocation decision, and Flamingo demonstrated few-shot learning across interleaved text and images.
Embodied systems widened their action vocabulary. RT-1 trained a transformer across hundreds of real robot tasks, and Gato placed text, images, control observations, and actions in one autoregressive stream. AlphaTensor pushed RL beyond game playing into the discovery of correct, hardware-tuned matrix multiplication procedures.
Media and systems changed together: Whisper used large-scale weak supervision for robust speech recognition; AudioLM modeled audio through discrete representations; and Video Diffusion Models extended denoising through time. FlashAttention made attention IO-aware, proving that an exact algorithm can be much faster simply by moving fewer bytes.
2023: open adaptation met production serving
LLaMA accelerated work on openly available foundation models. QLoRA combined a quantized frozen base with LoRA adapters, and DPO replaced an explicit reward-model-plus-RL loop with a direct preference objective. Together they made data curation, evaluation, and adaptation recipes more differentiating than access to a single base checkpoint.
Robotics connected web knowledge to action. PaLM-E injected continuous embodied observations into a language model; RT-2 expressed actions as tokens in a vision-language-action model; and Open X-Embodiment pooled data across robot types. The enduring bottleneck became clear: model scale helps, but embodiment-specific data, action interfaces, latency, and evaluation still determine whether a policy works.
Closed-loop learning and systems evaluation also matured. DreamerV3 used learned dynamics and imagined rollouts across diverse domains, while AlphaDev searched for faster sorting routines. In software, SWE-bench moved evaluation from short code snippets to real repository issues. At inference time, vLLM/PagedAttention attacked KV-cache fragmentation and AWQ made low-bit weight-only serving accuracy-aware.
2024: agents and world models became systems
Robot learning gained reusable policies and predictive representations. OpenVLA released an open vision-language-action model, Octo trained a generalist policy across multiple embodiments, and V-JEPA predicted masked video representations rather than reconstructing every pixel. These are complementary bets: a policy maps observation to action; a world model predicts consequences so an agent can plan before acting.
Sora’s technical report scaled a transformer over spacetime patches, while documenting failures in physics, object state, and long-duration coherence. That combination is historically useful: visual quality rose quickly, but controllable production still required shot boundaries, references, editing, and continuity checks.
RL re-entered language-model post-training through objectives with checkable feedback. DeepSeekMath introduced Group Relative Policy Optimization (GRPO), and AlphaGeometry paired a neural language model with symbolic deduction. SWE-agent showed that an agent’s computer interface and trajectory design materially affect repository-level coding. The “model call” was no longer the product; the scaffold around repeated calls was.
2025: prediction, search, and action converged
DeepSeek-R1 made reinforcement learning and verifiable tasks central to an open reasoning-model recipe. The larger shift was from paying only for a larger forward pass to allocating compute across generation, checking, search, and revision. That same loop appeared in software: OpenAI’s cloud Codex release placed repository work in isolated environments and made parallel tasks and proposed changes part of the interface.
In embodied AI, V-JEPA 2 combined large-scale self-supervised video pretraining with a smaller amount of robot data for action-conditioned planning, while GR00T N1 presented a generalist humanoid foundation model. Both point toward the same data strategy: abundant observation can teach reusable structure, but grounded interaction is still needed to connect that structure to an embodiment.
Media models also crossed a modality boundary. Google announced Veo 3 with native audio generation in May 2025, bringing dialogue, environmental sound, and picture into one generation path. That increased the need for synchronized evaluation and provenance; it did not remove the editor’s need to control performance, timing, sound perspective, and continuity across shots.
January–July 2026: architecture matters more than the release name
By mid-2026, frontier releases increasingly packaged sparse activation, native multimodality, long context, tool use, and agent-oriented training together. Moonshot’s dated Kimi K3 announcement is one concrete example, not a claim that the family is permanently “best.” The important systems question is which weights can actually be deployed, adapted, evaluated, and served within a latency, privacy, and cost envelope.
Robotics research also began stating the missing interfaces more explicitly. The June 2026 position paper Robots Need More than VLA and World Models argues for data, embodiment, physics-grounded world-model, and reward interfaces. That is a research agenda rather than a settled result, but it captures the engineering gap: a general policy cannot consume unstructured human video, retarget actions, infer progress, and respect robot dynamics without surrounding systems.
The same is true in coding and media. Multi-agent harnesses need ownership boundaries, sandboxes, tests, provenance, and a reviewer; generative-film systems need asset state, shot graphs, edit decisions, and selective regeneration. Inference companies such as Etched and MatX publicly frame their work as workload-specific chip, system, and software co-design. Marketing claims still require independent measurement, but the direction follows five years of evidence: architecture, kernels, memory, scheduling, and model structure must be optimized together.
What endured
- A common sequence interface is leverage, not magic. Text, image patches, audio codes, actions, and tool calls can share modeling machinery; geometry, physics, and domain constraints still need explicit structure.
- Post-training is a first-class system. Supervised data, preferences, verifiers, RL, and retention tests determine behavior after broad pretraining has created capability.
- Closed loops beat one-shot generation. Agents, robots, world models, and decision systems all improve when they can act, inspect state, score outcomes, and revise.
- Evaluation migrated toward the real environment. Repository issues, compilers, hardware runtime, real robot rollouts, and human preference reveal failures hidden by proxy metrics.
- Memory movement is part of model design. FlashAttention, KV-cache paging, quantization, sparsity, and workload-specific accelerators made serving architecture inseparable from model architecture.
- Hybrid systems remain the durable product pattern. Learned components work best beside retrieval, planning, simulators, symbolic checks, editors, and deterministic software that expose control and failure.
What I would build from this history
For robotics, I would connect episode annotation, simulation, world-model prediction, a VLA or imitation policy, and paired rollouts with a failure taxonomy. For RL and market decision systems, I would make the simulator, point-in-time data, uncertainty, policy baseline, and counterfactual evaluation visible. For model work, I would compare full fine-tuning, LoRA/QLoRA, preference optimization, quantization, and serving on the same task and retention suite.
For coding agents, I would build the harness: typed tasks, bounded tools, isolated worktrees, event logs, tests, independent review, and recovery from partial failure. For accelerated computing, I would climb from reductions and tiled GEMM to attention, quantization, KV-cache operations, and complete transformer blocks, measuring bandwidth, occupancy, numerical error, and end-to-end model impact rather than collecting kernels as isolated puzzles.
This history will keep changing. The synthesis should be revised when a new result changes an architectural conclusion, not every time a new checkpoint moves a benchmark by a few points.