Natural language understanding: semantics, grounding, and honest evaluation

Why this subject matters now

Five years ago a practitioner could get a long way treating an NLP benchmark as ground truth. Pick a leaderboard, train until the number goes up, ship. That habit is now a liability. The last few years produced a string of results showing that headline accuracy on the most-cited datasets was inflated by statistical shortcuts the model could exploit without doing the task. Natural language inference models were beating strong baselines while looking only at the hypothesis and never reading the premise. Reading-comprehension systems were answering questions correctly with the passage deleted. Models that scored in the high nineties on a test set collapsed to near chance on a hand-edited contrast set that changed the answer with a minimal perturbation. And as models were trained on ever larger crawls of the web, the test sets themselves started leaking into training data, so a rising number could mean memorization rather than generalization. Every one of these is a measurement failure, not a modeling failure, and a senior practitioner is now expected to diagnose them.

The second thing that changed is that representations became the product. When the deliverable was a classifier, its internal states were nobody's business. When the deliverable is a frozen encoder whose embeddings feed retrieval, ranking, clustering, and a dozen downstream heads, the question of what those vectors encode is a first-class engineering concern. Probing is the tool that grew up to answer it, and probing has its own measurement problem. A sufficiently powerful probe can extract a signal that the representation does not actually make available in any useful sense, so a probe accuracy in isolation tells you almost nothing. The selectivity control, derived below, is the correct instrument.

Third, retrieval moved to the center. Dual-encoder retrievers trained with a contrastive objective now sit in front of most production question-answering and retrieval-augmented generation systems, and the metrics that govern them, recall@k and mean reciprocal rank, behave differently from classification accuracy in ways that trip people up. Understanding why a late-interaction model like ColBERT retrieves differently from a single-vector DPR encoder, and why fusion-in-decoder reads many passages instead of one, is now table stakes. This page derives the objective, computes the metrics, and connects them to the honesty question. A retriever that scores well on an i.i.d. split can still fail open-domain because the split never tested the thing that matters.

what a benchmark number can hide
  ↓
probe capacity      annotation artifact   i.i.d. split        contamination      miscalibration
(Hewitt & Liang)   (Gururangan et al.)   (Lake & Baroni)    (Recht et al.)     (Guo et al.)
  ↓                   ↓                     ↓                   ↓                  ↓
probe memorizes     hypothesis-only       trained and         test set leaked    confidence not
random labels       cue beats chance      tested on the       into pretraining   equal to accuracy
                                          same distribution
  ↓                   ↓                     ↓                   ↓                  ↓
selectivity         contrast sets,        SCAN / COGS         held-out, dated    ECE + temperature
control task        hypothesis-only       compositional       eval; decontam.    scaling
                    baseline reported     splits

What "understanding" is supposed to mean

Before measuring understanding it helps to be precise about the target, because a great deal of confusion comes from equating fluency with comprehension. The cleanest statement of the problem is due to Bender and Koller (2020), who separate form from meaning. Form is the observable text, the sequence of tokens. Meaning is the relation between form and something external to the text, the communicative intents and the states of affairs in the world that the text is about. A system trained only on form has access to the joint distribution of tokens and nothing else. The distributional signal is rich enough to recover an enormous amount of structure, syntax, selectional preferences, rough analogical relations, but it is, in the strong version of the argument, in principle unable to recover reference, which entity in the world a name picks out, whether a described situation actually holds. That is what grounding supplies, a link from symbols to perception, action, or a knowledge base.

One does not have to accept the strongest form of the claim to take the operational lesson, which is that a benchmark built entirely out of text can only ever test form-internal competence. If a task can be solved by exploiting regularities of the text alone, then success on it is evidence about distributional learning, not about grounded understanding, and the burden is on the benchmark designer to show that the task cannot be solved that way. This is the thread that connects every section below. Probing tests what a representation makes linearly available, artifact analysis tests whether a task can be shortcut, contrast sets test whether a decision tracks meaning or surface form, and contamination analysis tests whether a number reflects generalization or recall. Grounding proper, through images, video, embodiment, or tool use, is the subject of the multimodal foundation models page. Here the concern is how to measure honestly when the only signal is text.

Contextual representations and what probing measures

A contextual encoder maps a token in context to a vector. For a sentence of tokens \(w_1,\dots,w_T\), layer \(\ell\) produces hidden states \(h^{(\ell)}_1,\dots,h^{(\ell)}_T\) with \(h^{(\ell)}_t \in \R^d\). Unlike a static word vector, \(h^{(\ell)}_t\) depends on the whole sentence, so the same word type gets different vectors in "river bank" and "bank loan". The empirical question that launched a field is what linguistic information is present in \(h^{(\ell)}_t\), and where. The dominant tool is the probing classifier, sometimes called a diagnostic classifier. Freeze the encoder and train a small supervised model \(g_\phi\) to predict a linguistic property (part of speech, dependency label, coreference, semantic role) from the hidden state. If \(g_\phi\) reaches high accuracy, the property is said to be encoded.

What a probe actually estimates

Fix a probing task with input the frozen vector \(h\) and target label \(y\) drawn from a joint distribution induced by the corpus and the encoder. A probe is a function class \(\mathcal{G}\), and probing reports

estimated on held-out data. The quantity being maximized is the accuracy of the best predictor in the class. Two things follow immediately. First, this is an existence claim about a decoder, not a claim that the encoder or any downstream user actually reads the property this way. Second, and this is the confound that defines the field, the reported number depends on \(\mathcal{G}\) as much as on \(h\). If \(\mathcal{G}\) is expressive enough to fit arbitrary maps from \(h\) to \(y\), then a high accuracy is guaranteed as long as \(h\) merely identifies the token, because the probe can memorize a lookup table from vectors to labels. High probe accuracy is then evidence that the encoder preserves token identity, which every non-degenerate encoder does, and evidence about the property is confounded with evidence about probe capacity.

Tenney, Das, and Pavlick (2019) and Tenney et al. (2019, "BERT rediscovers the classical NLP pipeline") made probing quantitative with edge probing, casting many tasks (constituents, dependencies, entities, semantic roles, coreference, relations) as classification over spans of hidden states and reading off which layers carry which signal. The headline finding, that the layerwise center of gravity for a task follows the order of a classical pipeline, part-of-speech low, then constituents, then dependencies, then semantic roles and coreference higher up, is one of the most cited results in interpretability. It is also exactly the kind of result that the capacity confound can distort, because a more expressive probe shifts the apparent location of information. The fix is not a better probe. It is a control.

The control task and selectivity

Hewitt and Liang (2019) diagnosed the confound and proposed the standard against which probe results are now judged. Alongside the real linguistic task, construct a control task with the same input and output structure but with the true structure destroyed. Assign each word type a fixed, random label from the same label set, independently of context. A control task has, by construction, no linguistic content. The only way to fit it is to memorize the random type-to-label map. Now train the same probe on both tasks and report two accuracies. Define

The logic is a difference-in-differences. A high-capacity probe scores well on both tasks, because it can memorize, so its selectivity is small. A probe that scores well on the linguistic task while scoring poorly on the control task cannot be memorizing, so its accuracy must come from structure the representation actually makes available. Selectivity, not accuracy, is the quantity that isolates the representation from the probe. The practical recipe is to prefer the smallest probe (often a linear map or a one-hidden-layer network with a tight bottleneck, controlled by hidden size, dropout, or weight decay) that keeps linguistic accuracy high, because that is the probe with the highest selectivity, and to report selectivity alongside accuracy always.

The structural probe, syntax as geometry

Edge probing asks whether a label is decodable. Hewitt and Manning (2019) asked a sharper geometric question, whether an entire dependency parse tree is embedded in the representation as distance, and if so, under what metric. Their structural probe looks for a single linear transformation \(B \in \R^{k \times d}\) such that, after applying \(B\), the squared Euclidean distance between two word vectors matches the distance between those two words in the parse tree, where tree distance \(d_T(i,j)\) is the number of edges on the path between words \(i\) and \(j\).

Write \(h_i\) for the hidden state of word \(i\). The probe defines a squared distance in the transformed space,

which is a squared Mahalanobis distance with positive-semidefinite matrix \(A = B^\top B\). The geometry of why squared distance is the right target is the crucial point. A tree metric \(d_T\) does not embed isometrically into Euclidean space, but the map \((i,j) \mapsto d_T(i,j)\) behaves like a squared Euclidean distance in a precise sense. Place each node of a tree at a point so that every edge is an orthogonal unit step, that is, give the tree with \(n\) nodes an embedding into \(\R^{n-1}\) where each edge points along a fresh coordinate axis. Then two nodes separated by \(d_T(i,j)\) edges differ in exactly \(d_T(i,j)\) coordinates by one unit each, so their squared Euclidean distance is exactly \(d_T(i,j)\). Tree distance is intrinsically a squared Euclidean quantity, which is why the probe fits \(d_B(\cdot)^2\), not \(d_B(\cdot)\), to \(d_T\).

The probe is trained by minimizing, over all sentences and all word pairs, the average absolute error between predicted and true tree distance,

with \(T\) the sentence length used to normalize across lengths. This is a simple linear-algebra regression, and its success is a strong statement. It says a low-dimensional linear subspace of the representation carries the parse tree as squared distance, recoverable without any nonlinearity. A companion construction recovers tree depth as the squared norm \(\lVert B h_i \rVert^2\). The evaluation is not the raw regression loss but two tree-structural metrics, the undirected unlabeled attachment score of the minimum spanning tree induced by the predicted distances and the Spearman correlation between predicted and true depths. Reporting a geometric fit as a parsing score keeps the claim honest, since a low regression loss that does not yield a correct tree would be a probe artifact rather than a property of the representation.

Natural language inference and the annotation-artifact problem

Natural language inference, also called recognizing textual entailment, is the task of deciding, given a premise sentence and a hypothesis sentence, whether the premise entails the hypothesis, contradicts it, or is neutral. It was proposed as a clean probe of understanding, since to decide entailment one seemingly must represent the meaning of both sentences and the relation between them. Bowman et al. (2015) built SNLI, a large crowdsourced dataset of 570k pairs. Williams, Nangia, and Bowman (2018) built MultiNLI to add genre diversity. These datasets powered a generation of models and are still standard.

How the artifacts got in

The datasets were built by showing an annotator a premise and asking them to write, for each label, one hypothesis, an entailed sentence, a neutral one, and a contradictory one. This protocol is efficient and it is also the source of a systematic leak. Human annotators fall into habits. To make a sentence contradictory, the fastest move is negation, so words like "not", "no", and "never" appear far more in contradiction hypotheses than in others. To make a sentence entailed, a safe move is to generalize, so "animal", "outdoors", and "instrument" cluster in entailment. To make a sentence neutral, annotators add unsupported specifics, so "tall", "first", "because" cluster in neutral. None of these cues has anything to do with the premise. They are properties of the hypothesis alone, and they are correlated with the label. That correlation is an annotation artifact.

Gururangan et al. (2018) and, independently, Poliak et al. (2018) made the point plain with the hypothesis-only baseline, training a classifier that sees only the hypothesis and never the premise. On a task where the label is supposed to be a relation between two sentences, such a model should do no better than the majority class. Instead it substantially beat the majority baseline on SNLI and MNLI. A model reading only half the input, the half that logically cannot determine the answer, was extracting real signal, which means a large fraction of the "understanding" a full model appeared to demonstrate was the same shortcut. The remedy the field adopted is procedural. Always report the hypothesis-only (and premise-only) baseline for any new inference dataset, and treat the gap between the full model and the partial-input baseline, not the full-model accuracy, as the measure of how much the task demands the relation.

Quantifying the leak with mutual information

The right language for "a cue carries information about the label" is information theory. Let \(X\) be a binary feature of the hypothesis, say \(X = 1\) if it contains a negation word, and let \(Y\) be the three-way label. The pointwise mutual information of a specific co-occurrence is

and the mutual information, the expected PMI, measures the total dependence between cue and label,

\(I(X;Y) = 0\) exactly when the cue is independent of the label, which is what a leak-free dataset would give. Any positive value is a quantified shortcut. It upper-bounds nothing about a model's behavior directly, but it certifies that a hypothesis-only classifier has exploitable signal, and its magnitude in bits is comparable across cues and datasets. The next problem computes both the hypothesis-only accuracy and \(I(X;Y)\) on a toy dataset engineered to look like SNLI's negation artifact.

Compositional generalization, or why i.i.d. splits flatter models

The standard train/test split draws both sets from the same distribution. If a model has memorized shortcuts specific to that distribution, an i.i.d. test set will not catch it, because the test set contains the same shortcuts. Compositional generalization is the ability to understand novel combinations of known parts, "jump twice" from having seen "jump" and "walk twice", and it is exactly what an i.i.d. split fails to probe, since novel combinations are, by definition, out of the training distribution. Two datasets made this measurable by construction.

SCAN (Lake and Baroni, 2018) maps command strings to action sequences in a tiny synthetic language, where "jump twice" becomes JUMP JUMP and "walk left and jump" becomes LTURN WALK JUMP. The point is the splits. Under a random i.i.d. split, sequence-to-sequence RNNs solve it near perfectly. Under the add-primitive split, where "jump" appears in training only in isolation and every composed command with "jump" is held out for test, the same models collapse, often below 10% exact-match, because they never learned that "twice" is a function that applies to any primitive, they learned the specific training combinations. Under the length split, where test sequences are longer than any seen in training, they fail again. The i.i.d. number was near 100% while the compositional number was near zero. The same model, two splits, opposite conclusions about whether it understood the grammar.

COGS (Kim and Linzen, 2020) raised the bar to semantic parsing over naturalistic English, mapping sentences to logical forms, with a generalization set built from systematic gaps. A noun seen only as a subject must be interpreted as an object, a structure seen only at shallow depth must be handled at greater depth, and so on. Twenty-one distinct generalization types, each a named compositional gap. Strong models that scored above 95% on the i.i.d. test set dropped by tens of points on the generalization set. The lesson is methodological and it generalizes far beyond these two datasets. A single i.i.d. number cannot certify compositional ability, and any claim of understanding must be tested on a split that holds out combinations, not just examples. This is the same disease as the NLI artifact seen from the split side rather than the feature side.

Retrieval and open-domain question answering

Open-domain question answering removes the assumption that the relevant passage is handed to the model. Given a question and a corpus of millions of passages, the system must first retrieve a small set of candidate passages and then read them to produce an answer. Retrieval quality caps everything downstream. If the answer-bearing passage is never retrieved, no reader can recover it. For two decades retrieval meant sparse lexical matching, BM25 over an inverted index, which is a strong baseline but blind to synonymy and paraphrase. Dense retrieval learns the matching function instead.

The dual encoder and maximum inner-product search

A dual encoder, or bi-encoder, has two networks, a question encoder \(E_Q\) and a passage encoder \(E_P\), each mapping text to a vector in \(\R^d\). The relevance score of passage \(p\) to question \(q\) is their inner product,

Retrieval is then \(\argmax_p s(q,p)\) over the corpus, and top-\(k\) retrieval is the \(k\) passages of highest inner product, a maximum inner-product search (MIPS). The decisive engineering property is that passage vectors do not depend on the question, so the entire corpus can be encoded once, offline, into a matrix \(P \in \R^{M \times d}\), indexed with an approximate-nearest-neighbor structure (FAISS with HNSW or IVF-PQ). At query time only the question is encoded and a single approximate MIPS returns candidates in milliseconds against millions of passages. This is exactly the DPR architecture of Karpukhin et al. (2020). To use cosine similarity instead, normalize both vectors first. Inner product on normalized vectors is cosine, and inner product on unnormalized vectors lets the model encode a notion of passage "prior importance" in the norm.

The contrastive objective, derived

The retriever is trained so that a question's vector is closer to its gold passage than to irrelevant passages. Given a batch of \(B\) question-passage pairs \((q_i, p_i^+)\), each with its correct passage, form the similarity matrix \(S \in \R^{B \times B}\) with \(S_{ij} = s(q_i, p_j)/\tau\), where \(\tau\) is a temperature. The diagonal entries are the positives. Every off-diagonal entry \(S_{ij}\ (j \ne i)\) is an in-batch negative, the gold passage of a different question, reused for free as a negative for this one. The loss is the cross-entropy of a softmax over passages, treating retrieval as \(B\)-way classification of which passage matches the question,

This is the InfoNCE objective. It is worth seeing why it does the right thing by differentiating one term. Write \(\sigma_i \in \R^{B}\) for the softmax of row \(i\), so \(\sigma_{ij} = \frac{\exp(S_{ij})}{\sum_k \exp(S_{ik})}\). Using the standard softmax-cross-entropy gradient with target index \(i\),

and since \(S_{ij} = s(q_i,p_j)/\tau\), the gradient with respect to the question vector is

The update pulls the question vector toward its gold passage with strength proportional to how much probability mass the model is still missing on the positive, \(1 - \sigma_{ii}\), and pushes it away from each negative in proportion to the mass it wrongly assigns there, \(\sigma_{ij}\). At the optimum the model concentrates the row's probability on the diagonal. The temperature \(\tau\) scales the whole gradient. Small \(\tau\) sharpens the softmax and makes the loss dominated by the hardest negatives, while large \(\tau\) softens it. In-batch negatives make the objective nearly free, one matrix multiply \(Q P^\top\) gives all \(B^2\) scores, and larger batches supply more negatives, which is why dense retrieval training benefits from large batches. DPR further adds one hard negative per question, typically a BM25-retrieved passage that is lexically similar but wrong, because random in-batch negatives are usually too easy.

Late interaction and fusion-in-decoder

A single-vector dual encoder compresses a whole passage into one \(d\)-dimensional vector, which is lossy, since fine-grained term matches get averaged away. Two influential designs relax this. ColBERT (Khattab and Zaharia, 2020) keeps a vector per token for both query and passage and defines relevance by late interaction. For each query token, take its maximum similarity over all passage tokens, then sum over query tokens,

This recovers term-level matching, a query token that must appear finds its best passage match, while keeping the passage representations precomputable, since the max-sim is evaluated at query time but the passage token vectors are indexed offline. It sits between the cheap single-vector retriever and an expensive cross-encoder that jointly attends over the concatenated query and passage. Fusion-in-decoder (Izacard and Grave, 2021) attacks the reading side. Retrieve many passages, encode each independently with the question (so cost is linear, not quadratic, in the number of passages), then let the decoder attend jointly over the concatenation of all encoded passages when generating the answer. The asymmetry is deliberate, independent encoding keeps the encoder cheap and parallel, while joint decoding lets the model aggregate evidence spread across passages, which is what open-domain answers frequently require.

Metrics that behave differently from accuracy

Retrieval is judged by rank-sensitive metrics, not accuracy. Recall@k is the fraction of queries for which a relevant passage appears in the top \(k\). It is monotone increasing in \(k\) and says nothing about position within the top \(k\). Mean reciprocal rank rewards putting the relevant passage high. For each query with first relevant result at rank \(r\), the reciprocal rank is \(1/r\) (and \(0\) if none is retrieved), and MRR is the mean over queries,

The two can disagree, and understanding the disagreement is the point. Recall@k is the metric that matters when a downstream reader will consume all \(k\) passages regardless of order, as fusion-in-decoder does. MRR is the metric that matters when only the top result is used or when higher rank is genuinely better. Optimizing recall@100 and optimizing MRR pull a retriever in different directions, and reporting one while a system is judged on the other is a common and quiet mistake.

Calibration, or when a confidence is a probability

A classifier outputs not just a prediction but a confidence, the probability it assigns to its top class. That confidence is calibrated if it matches the empirical frequency of being correct. Among all predictions made with confidence \(0.8\), exactly 80% should be right. Calibration is distinct from accuracy. A model can be accurate but overconfident, right 90% of the time while claiming 99%, which is exactly the failure mode Guo et al. (2017) documented for modern neural networks. Unlike the shallow models of the 1990s, deep networks trained with the usual recipe are systematically overconfident, and the miscalibration grows with capacity. For any system whose confidence feeds a decision, an abstention threshold, a routing choice, a downstream Bayesian combination, calibration is not optional.

Expected calibration error, derived

The population quantity of interest is the average gap between confidence and correctness. Let \(\hat p\) be the model's top-class confidence and let \(C\) be the indicator of a correct prediction. Perfect calibration means \(\P(C = 1 \mid \hat p = c) = c\) for all \(c\). The expected calibration error is the expected absolute deviation from that line,

Since \(\hat p\) is continuous, this expectation cannot be estimated pointwise, so it is approximated by binning. Partition \([0,1]\) into \(M\) equal bins \(B_1,\dots,B_M\), and in each bin compare the average confidence to the empirical accuracy,

Each term weights a bin's calibration gap by how many predictions fall in it, so a large gap in a rarely used confidence range contributes little. The estimator is biased, it depends on \(M\), and it can hide compensating errors within a bin, which is why the reliability diagram, a bar chart of accuracy versus confidence per bin, is reported alongside the scalar. But as a single number it is the field standard.

Temperature scaling, the one-parameter fix

The cheapest effective recalibration is temperature scaling, the single-parameter special case of Platt scaling that Guo et al. (2017) showed is usually enough. After training, freeze the network and introduce one scalar \(T > 0\) that divides the logits before the softmax,

\(T^\star\) is fit by minimizing negative log-likelihood on a held-out validation set, a one-dimensional convex-in-practice search. The decisive property is that temperature scaling cannot change the prediction. Dividing every logit by the same positive \(T\) preserves their order, so \(\argmax_i z_i/T = \argmax_i z_i\), and accuracy is exactly unchanged. It only rescales the gap between logits, which is what the softmax turns into confidence. \(T > 1\) softens an overconfident model, spreading probability toward other classes, while \(T < 1\) sharpens an underconfident one. The next problem makes the invariance concrete.

Adversarial data, contrast sets, and behavioral testing

The failures above are all detectable by holding out the right thing. But a fixed test set, however well constructed, is a static target, and models overfit to targets. Three complementary methods harden evaluation against this.

Contrast sets (Gardner et al., 2020) fix the local decision boundary. The dataset creators take existing test examples and make small, meaning-changing edits, the minimal perturbation that flips the gold label, producing clusters of examples that are close in input space but span the decision boundary. A model that has learned the task should get the whole cluster right. A model that has learned a shortcut correlated with the label in the original distribution will get the original right and its perturbed neighbor wrong. The reported metric is contrast consistency, the fraction of clusters answered entirely correctly, and it is routinely tens of points below standard accuracy. Contrast sets probe the model's decision boundary where it actually is, rather than where the i.i.d. distribution happens to sample.

Behavioral testing (CheckList, Ribeiro et al., 2020) borrows software engineering's idea of a test suite. Rather than one aggregate accuracy, enumerate capabilities the task requires, negation, coreference, robustness to typos, temporal ordering, fairness across swapped names, and write targeted tests for each, using three test types, a minimum functionality test (simple examples isolating one capability), an invariance test (perturbations that must not change the label, like a synonym swap), and a directional expectation test (perturbations that must change the label in a known direction). The output is a matrix of failure rates by capability, which localizes weakness rather than averaging it away. CheckList found high-scoring commercial sentiment and NLI systems failing simple negation and coreference tests at rates a single accuracy number hid.

Adversarial and human-in-the-loop collection closes the loop by having annotators, sometimes aided by a model-in-the-loop that filters out examples the current model already gets right, deliberately author examples that fool the best available system. The resulting datasets are harder by construction, though they carry their own bias, they overrepresent the current model's specific weaknesses, and a dataset built to break one model may be easy for the next. All three methods share a philosophy, to stop trusting a single held-out number and instead characterize where and how a model fails.

The measurement problem, building a number that survives a new model

The deepest issue is not any single artifact but the reliability of benchmark numbers as models improve. Three threats matter in practice.

Contamination. As pretraining corpora grew to trillions of tokens scraped from the web, the public test sets, which live on the web, began to appear in the training data. A model that has seen the test questions during pretraining can recall answers rather than solve tasks, and the reported number then measures memorization. The defenses are decontamination (n-gram overlap filtering of training data against evaluation sets, which is imperfect because paraphrases evade it), held-out and freshly-collected evaluation, and dating evaluation sets so that any set predating a model's data cutoff is treated with suspicion. A benchmark whose examples are public and static is on a clock. Its numbers degrade in meaning the moment it is popular enough to be crawled.

Adaptive overfitting. When a community evaluates thousands of models against one test set, choosing architectures and hyperparameters by test performance, the test set is no longer held out, it has been used, indirectly, for selection. The classic study is Recht et al. (2019), who built fresh test sets for CIFAR-10 and ImageNet by replicating the original collection pipeline and re-measured the same models. Accuracies dropped, several points, but crucially the ranking of models was largely preserved and the drop was smooth, which was reassuring. The community had not catastrophically overfit, but the absolute numbers on the original test set were optimistic. The lesson for NLU is that a leaderboard's absolute number is inflated by the collective search that produced it, and only a fresh test set measures the real thing.

Construct validity. Even a clean, uncontaminated benchmark measures what it measures, which may not be the construct its name claims. "Natural language inference" is operationalized as a three-way classification over a specific dataset, and a high score certifies competence on that operationalization, not on inference in general, especially when the operationalization leaks cues. Construct validity is the question of whether the measurement instrument actually measures the target capability, and it is the question the whole page has been circling. Probing without a control lacks construct validity because it measures probe capacity. NLI accuracy without a hypothesis-only baseline lacks it because it measures a shortcut. An i.i.d. split lacks it because it measures in-distribution recall. A number survives a new model when the task cannot be shortcut, the split holds out the right structure, the data is uncontaminated, and the metric matches the use. Building such a number is the actual work of natural language understanding evaluation, and it is harder than training the model.

Implementation

The first block implements the dual-encoder retrieval scorer and its in-batch contrastive loss in PyTorch and JAX. Both encode a batch of questions and passages, L2-normalize (so the inner product is cosine), build the full \(B \times B\) similarity matrix with one matrix multiply, and apply cross-entropy against the diagonal targets, exactly the InfoNCE loss derived above. The tensors are annotated with shapes.

import torch
import torch.nn as nn
import torch.nn.functional as F

class DualEncoder(nn.Module):
    """Two independent encoders; relevance is the inner product of their outputs."""
    def __init__(self, q_encoder: nn.Module, p_encoder: nn.Module, tau: float = 0.05):
        super().__init__()
        self.q_encoder = q_encoder      # maps question tokens -> (B, d)
        self.p_encoder = p_encoder      # maps passage tokens  -> (B, d)
        self.tau = tau

    def score(self, q_emb, p_emb):      # q_emb: (B, d), p_emb: (M, d)
        q = F.normalize(q_emb, dim=-1)  # cosine <=> inner product on unit vectors
        p = F.normalize(p_emb, dim=-1)
        return q @ p.t()                # (B, M) similarity matrix

    def in_batch_loss(self, q_emb, p_emb):
        # q_emb, p_emb: (B, d); passage i is the gold for question i (diagonal)
        S = self.score(q_emb, p_emb) / self.tau     # (B, B), logits
        targets = torch.arange(S.size(0), device=S.device)
        # cross-entropy over passages == InfoNCE with in-batch negatives
        return F.cross_entropy(S, targets)

# --- toy check against the worked problem (tau = 1) ---
S = torch.tensor([[3.0, 1.0, 0.5],
                  [0.8, 2.5, 1.2],
                  [0.2, 0.9, 2.0]])
targets = torch.arange(3)
loss = F.cross_entropy(S, targets)     # tau folded into S already
print(float(loss))                     # -> 0.3254, matches the hand computation

def recall_at_k(scores, gold_idx, k):  # scores: (B, M), gold_idx: (B,)
    topk = scores.topk(k, dim=-1).indices          # (B, k)
    hit = (topk == gold_idx[:, None]).any(dim=-1)  # (B,)
    return hit.float().mean().item()

def mrr(scores, gold_idx):
    order = scores.argsort(dim=-1, descending=True)          # (B, M)
    ranks = (order == gold_idx[:, None]).float().argmax(-1)  # 0-based rank of gold
    return (1.0 / (ranks + 1)).mean().item()

import jax
import jax.numpy as jnp

def normalize(x, eps=1e-12):
    return x / (jnp.linalg.norm(x, axis=-1, keepdims=True) + eps)

def score(q_emb, p_emb):               # q_emb: (B, d), p_emb: (M, d)
    return normalize(q_emb) @ normalize(p_emb).T    # (B, M)

def in_batch_loss(q_emb, p_emb, tau=0.05):
    # passage i is the gold for question i (the diagonal of S)
    S = score(q_emb, p_emb) / tau                   # (B, B) logits
    logp = jax.nn.log_softmax(S, axis=-1)           # row-softmax over passages
    B = S.shape[0]
    return -jnp.mean(jnp.diagonal(logp))            # InfoNCE, in-batch negatives

# --- toy check against the worked problem (tau = 1) ---
S = jnp.array([[3.0, 1.0, 0.5],
               [0.8, 2.5, 1.2],
               [0.2, 0.9, 2.0]])
logp = jax.nn.log_softmax(S, axis=-1)
print(float(-jnp.mean(jnp.diagonal(logp))))        # -> 0.3254

def recall_at_k(scores, gold_idx, k):  # scores: (B, M), gold_idx: (B,)
    topk = jax.lax.top_k(scores, k)[1]             # (B, k) indices
    hit = jnp.any(topk == gold_idx[:, None], axis=-1)
    return jnp.mean(hit.astype(jnp.float32))

def mrr(scores, gold_idx):
    order = jnp.argsort(-scores, axis=-1)                    # (B, M)
    ranks = jnp.argmax(order == gold_idx[:, None], axis=-1)  # 0-based
    return jnp.mean(1.0 / (ranks + 1))

The second block is pure Python (NumPy) and reproduces the two measurement computations exactly as worked by hand, the expected calibration error with equal-width binning and the annotation-artifact analysis, hypothesis-only accuracy and the mutual information between a lexical cue and the label. Both print the numbers derived in the problems above.

import numpy as np

def expected_calibration_error(conf, correct, n_bins=5):
    """conf: (n,) top-class confidences; correct: (n,) 0/1. Returns binned ECE."""
    conf = np.asarray(conf, float)
    correct = np.asarray(correct, float)
    edges = np.linspace(0.0, 1.0, n_bins + 1)
    n = len(conf)
    ece = 0.0
    for i in range(n_bins):
        lo, hi = edges[i], edges[i + 1]
        mask = (conf > lo) & (conf <= hi) if i > 0 else (conf >= lo) & (conf <= hi)
        if mask.sum() == 0:
            continue
        acc = correct[mask].mean()
        avg_conf = conf[mask].mean()
        ece += (mask.sum() / n) * abs(acc - avg_conf)
    return ece

conf =    [0.55, 0.62, 0.68, 0.71, 0.77, 0.83, 0.88, 0.91, 0.95, 0.98]
correct = [0,    1,    1,    0,    1,    1,    1,    1,    1,    1]
print("ECE =", round(expected_calibration_error(conf, correct, 5), 4))  # 0.122

def hypothesis_only(counts, labels):
    """counts[(x, y)] over binary cue x in {0,1} and label y. Best cue-only accuracy."""
    N = sum(counts.values())
    correct = 0
    for x in (0, 1):
        best = max(labels, key=lambda y: counts[(x, y)])
        correct += counts[(x, best)]
    return correct / N

def mutual_information(counts, labels):
    N = sum(counts.values())
    px = {x: sum(counts[(x, y)] for y in labels) / N for x in (0, 1)}
    py = {y: sum(counts[(x, y)] for x in (0, 1)) / N for y in labels}
    mi = 0.0
    for x in (0, 1):
        for y in labels:
            pxy = counts[(x, y)] / N
            if pxy > 0:
                mi += pxy * np.log2(pxy / (px[x] * py[y]))
    return mi

labels = ["E", "N", "C"]
counts = {(1, "C"): 150, (1, "N"): 20,  (1, "E"): 10,
          (0, "C"): 180, (0, "N"): 313, (0, "E"): 327}
print("hyp-only acc  =", round(hypothesis_only(counts, labels), 4))  # 0.477
print("majority base =", round(337 / 1000, 4))                       # 0.337
print("I(cue; label) =", round(mutual_information(counts, labels), 4))  # 0.178 bits

How it is done in practice

A production retrieval-augmented QA stack layers the ideas above. Passages are chunked (often a few hundred tokens with overlap), encoded once by the passage tower, and stored in a vector index. FAISS is the default, with HNSW graphs for low-latency in-memory search or IVF-PQ for compressed billion-scale indexes. At query time the question tower encodes the query, an approximate MIPS returns a few hundred candidates, and a heavier cross-encoder reranker re-scores the shortlist, since a cross-encoder that jointly attends over query and passage is far more accurate than a bi-encoder but too slow to run over the whole corpus. The reader, increasingly a decoder-only LLM rather than a dedicated fusion-in-decoder model, then conditions on the top passages. Hybrid retrieval, combining dense scores with BM25, remains standard because lexical matching still wins on rare entities and exact strings that a dense encoder smooths over.

The evaluation discipline in a serious shop mirrors the theory. New inference or classification datasets ship with partial-input baselines. Retrieval systems report both recall@k and MRR and state which the product optimizes. Calibration is checked with a reliability diagram and, where a confidence gates a decision, corrected with temperature scaling fit on a held-out split. And any headline benchmark result is accompanied by a contamination check against the training corpus. The open-source evaluation harnesses have absorbed these lessons. They standardize prompting, fix few-shot examples, and increasingly flag likely contamination, precisely so that a number reported by one group can be reproduced by another. The gap between a research accuracy and a deployed capability is almost always one of these measurement issues rather than a modeling one.

The current research frontier

Probing has matured into an argument about the right instrument. Beyond selectivity, the information-theoretic reframing of Voita and Titov (2020), which measures the description length of the labels given the representation rather than a fixed probe's accuracy, and the amnesic-probing and causal-intervention lines, which ask whether removing a property from the representation changes behavior rather than whether it is decodable, are the current best practice, because decodability and causal use are different questions. The Allen Institute, groups at MIT and Cornell, and researchers at Technion have pushed the causal framing hardest.

On evaluation, the frontier is dynamic and adversarial. Living benchmarks that refresh their test data, adversarial collection with a model in the loop, and the deliberate construction of held-out splits that stress specific compositional or reasoning gaps are replacing static leaderboards. The contamination problem has driven interest in canary strings, private test sets served only through an API, and post-hoc contamination detection. Construct-validity critiques from Bender, Koller, Raji, Bowman, and others have made "what does this benchmark actually measure" a mainstream question rather than a philosophical aside. On the retrieval side, the debate between single-vector, late-interaction, and learned-sparse representations, DPR against ColBERT against SPLADE, continues, with distillation from cross-encoder rerankers into bi-encoders and the training of general-purpose text embedding models (from Google, Microsoft, Cohere, and the BGE and E5 lines out of Beijing and Microsoft Research Asia) blurring the line between retrieval and representation learning. The through line across all of it is the theme of this page, that the community has stopped trusting a single number and started asking what would make a number trustworthy.

Open source to read

• facebookresearch/DPR is the reference dense passage retriever. Open dpr/models/biencoder.py to see the question and passage towers and the in-batch loss exactly as derived here, and train_dense_encoder.py for the hard-negative sampling.
• stanford-futuredata/ColBERT implements late interaction and the MaxSim operator. Read colbert/modeling/colbert.py for the per-token scoring and colbert/indexing/ for how per-token vectors are compressed and indexed at scale.
• huggingface/datasets and huggingface/evaluate host SNLI, MNLI, and the standard metrics. evaluate is the place to read a canonical, tested implementation of accuracy, and to see how metric modules are structured.
• EleutherAI/lm-evaluation-harness is the de facto standard harness for language-model evaluation. Read lm_eval/api/task.py for how a task fixes its prompt and scoring, which is where reproducibility and contamination questions are actually decided.
• allenai/allennlp and the associated contrast-set and CheckList releases are the reference implementations of behavioral and contrast-set testing. The CheckList repository marcotcr/checklist is where the three test types (MFT, INV, DIR) live.
• facebookresearch/faiss is the vector index behind essentially every dense-retrieval deployment. Start with the IndexFlatIP and IndexHNSWFlat wrappers to connect the MIPS math to the approximate structures used in practice.

Common misconceptions

"High probe accuracy proves the representation encodes the property." It proves a decoder in the probe's function class can extract it, which for a high-capacity probe is guaranteed by token identity alone. Without a control task and a selectivity number, a probe accuracy is uninterpretable, as Problem 1 shows, where two probes with the same 96% linguistic accuracy support opposite conclusions.

"A strong score on a benchmark means the model does the task." Not if the task can be shortcut. The hypothesis-only NLI baseline beats chance without reading the premise, and a reading-comprehension model can answer with the passage deleted. The honest quantity is the gap between the full model and the partial-input baseline, not the full-model score.

"Calibration is the same as accuracy." They are orthogonal. A model can be 90% accurate while claiming 99% confidence on every example, which is high accuracy and terrible calibration. Temperature scaling fixes the second without touching the first, precisely because dividing logits by a constant cannot reorder them.

"A random train/test split is a fair test of generalization." Only of in-distribution generalization. Compositional ability requires a split that holds out combinations, not examples. SCAN's add-primitive split turns a 99% model into a single-digit one without changing the model. An i.i.d. number cannot certify compositional understanding.

"Dense retrieval made BM25 obsolete." No. Lexical matching still wins on rare entities, exact strings, and out-of-domain queries that a dense encoder smooths over, which is why production systems run hybrid dense-plus-sparse retrieval and rerank. A single dense vector per passage is lossy, which is exactly the loss that late interaction was designed to recover.

"A leaderboard gain is a capability gain." It can be an artifact of contamination (the test set leaked into pretraining), of adaptive overfitting (thousands of models selected against one test set), or of a metric that flatters the system. A gain is a capability gain only when the task resists shortcuts, the data is uncontaminated, and the metric matches the use.

"Mutual information of zero means the feature is useless." For the specific label it means the feature is independent of that label, hence a hypothesis-only classifier gains nothing from it, which is the point of computing it. But zero MI with the label does not mean the feature is useless in combination with others. Artifacts are usually diagnosed one feature at a time precisely because the marginal dependence is the exploitable part.

Self-check

References

1. Jurafsky, D. and Martin, J. H. Speech and Language Processing, 3rd edition draft. The standard reference for semantics, inference, information retrieval, and evaluation, freely available online at the authors' page.
2. Bender, E. M. and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL. aclanthology.org/2020.acl-main.463.
3. Tenney, I., Das, D. and Pavlick, E. (2019). BERT Rediscovers the Classical NLP Pipeline. ACL. arXiv:1905.05950.
4. Tenney, I. et al. (2019). What do you learn from context? Probing for sentence structure in contextualized word representations (edge probing). ICLR. arXiv:1905.06316.
5. Hewitt, J. and Liang, P. (2019). Designing and Interpreting Probes with Control Tasks. EMNLP. arXiv:1909.03368.
6. Hewitt, J. and Manning, C. D. (2019). A Structural Probe for Finding Syntax in Word Representations. NAACL. aclanthology.org/N19-1419.
7. Voita, E. and Titov, I. (2020). Information-Theoretic Probing with Minimum Description Length. EMNLP. arXiv:2003.12298.
8. Bowman, S. R., Angeli, G., Potts, C. and Manning, C. D. (2015). A large annotated corpus for learning natural language inference (SNLI). EMNLP. arXiv:1508.05326.
9. Williams, A., Nangia, N. and Bowman, S. R. (2018). A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (MultiNLI). NAACL. arXiv:1704.05426.
10. Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S. and Smith, N. A. (2018). Annotation Artifacts in Natural Language Inference Data. NAACL. arXiv:1803.02324.
11. Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R. and Van Durme, B. (2018). Hypothesis Only Baselines in Natural Language Inference. *SEM. arXiv:1805.01042.
12. Lake, B. M. and Baroni, M. (2018). Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks (SCAN). ICML. arXiv:1711.00350.
13. Kim, N. and Linzen, T. (2020). COGS: A Compositional Generalization Challenge Based on Semantic Interpretation. EMNLP. arXiv:2010.05465.
14. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D. and Yih, W. (2020). Dense Passage Retrieval for Open-Domain Question Answering (DPR). EMNLP. arXiv:2004.04906.
15. Khattab, O. and Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR. arXiv:2004.12832.
16. Izacard, G. and Grave, E. (2021). Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (Fusion-in-Decoder). EACL. arXiv:2007.01282.
17. Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML. arXiv:1706.04599.
18. Gardner, M. et al. (2020). Evaluating Models' Local Decision Boundaries via Contrast Sets. Findings of EMNLP. arXiv:2004.02709.
19. Ribeiro, M. T., Wu, T., Guestrin, C. and Singh, S. (2020). Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. ACL. arXiv:2005.04118.
20. Recht, B., Roelofs, R., Schmidt, L. and Shankar, V. (2019). Do ImageNet Classifiers Generalize to ImageNet? ICML. arXiv:1902.10811.
21. McCoy, R. T., Pavlick, E. and Linzen, T. (2019). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (HANS). ACL. arXiv:1902.01007.
22. Belinkov, Y. and Glass, J. (2019). Analysis Methods in Neural Language Processing: A Survey. TACL. arXiv:1812.08951.