Our rule

5.1 The question, and the picture behind it

Assumes: Parts 3 and 4: a student and a teacher, tokens, SelfPatch's 3 × 3 window. Gives: the question of the runs in one sentence, and the picture of islands that the rule aims at.

The question, in the protocol's words: does a patch objective whose positives are the most similar patches anywhere in the crop, and whose negatives are the crop's least similar fifth, change the label-free mask quality, against the released start and the campaign's SelfPatch runs from the same start? Fact

Picture a photograph of a dog and a cat by a door. Cut into a grid of patches, the dog covers many patches, the cat a few others, and the door and the floor the rest. If the model's tokens worked as we hope, every patch of the dog would carry a token close to every other patch of the dog and far from the patches of the cat and the floor. The tokens would form islands of agreement, one island per object, and a decoder could read an object as an island. Toy

The image comes from GLOM, a paper that says it describes no working system: it proposes to represent the parts of an image by islands of identical vectors. This course uses GLOM as a picture, never as a method. Fact

SelfPatch, in Part 4, pulls each patch towards some of its 8 neighbours in its 3 × 3 window, and towards nothing else. Two things are missing for islands. A patch of the dog's ear can only agree with patches right next to it, so agreement creeps across an object one window at a time. And nothing in SelfPatch's loss ever pushes two patches apart: between patches it has attraction only. Fact

On 8 October 2026 the supervisor proposed, instead of SelfPatch's 3 × 3 window, to rank every patch against all other patches of the crop, take the top k as positives and the bottom k as negatives, do the same with a shared threshold for the image level, and change the loss accordingly. Fact

The owner then fixed the run in the chat of the same day: one run, with the row level and the image level combined, positives from the top k of all the patches, negatives from the lowest K per cent, and k fixed for the first run. For the loss the owner asked whether to keep SelfPatch's or to bring in a contrastive loss beside DINO's global loss; the lead recommended a contrastive loss on the tokens in place of SelfPatch's patch term, with DINO's global loss unchanged, and the owner proceeded. Fact

5.2 The rule, exactly as built

Assumes: 5.1, cosine similarity (Part 1), crops and the teacher (Part 3). Gives: every step and every constant of the rule, the code that computes it, and a worked example on a toy crop.

The rule runs on every crop of every image at every optimiser step. An image gives ten crops: two global crops of 224 pixels, each a grid of 14 × 14 = 196 patches, and eight local crops of 96 pixels, each a grid of 6 × 6 = 36 patches. Every patch of a crop is in turn the anchor, and the rule gives it two sets: its positives and its negatives. Fact

1. Similarity. Take the teacher's final-block tokens of the crop, scale each to length 1, and compute the cosine of every pair: a table of N × N numbers, the diagonal (a patch with itself) set to minus infinity so that it never counts. No gradient flows through this table. Fact
2. Two thresholds. Over the crop's N(N − 1) off-diagonal cosines, τ_hi is the quantile at 1 − k/(N − 1) and τ_lo the quantile at 0.20: one high and one low cut for the whole crop. Fact
3. Positives. The anchor's k most similar candidates (k = 8 on a 224-pixel crop, 4 on a 96-pixel crop), united with every candidate at or above τ_hi; above 24, the 24 most similar are kept. Fact
4. Negatives. The candidates at or below τ_lo that are not positives; above 48, the 48 least similar are kept. Fact
5. The term. For each positive, an InfoNCE term at temperature τ = 0.2 compares the student's token of the anchor with the teacher's tokens of the positive and of all the negatives. Fact
6. The objective. L = L_global + λ L_con with λ = 1.0; SelfPatch's patch term is not in it. Fact

Each step is one function of cluster/contrastive.py, and the trainer composes them crop by crop. Fact

The lab below is the first explainer's Lab N, unchanged in what it computes. At load it shows the worked example above: anchor 5 of the 4 × 4 toy crop under the combined rule at k = 4, the bottom 20 per cent and τ = 0.2. Click any patch to make it the anchor, switch to the 6 × 6 crop, or train the toy and watch the islands form. Toy

5.3 Real sets on a real photograph

Assumes: 5.2. Gives: the rule on the real teacher tokens of eight photographs and five models, with every count.

The lab runs the rule in your browser on the real tokens of the course's photographs: the 196 final-block tokens of each photograph's 224-pixel view, under five models (the converted DINO start, the released SelfPatch model, and the teachers of runs 1, 2 and 3 at their last epoch). Real

Start on the bulldog's nose (patch 34) under run 2. Its eight positives are its eight most similar patches, all within two patches of it, and all its negatives lie at least four rows below it. Then press "An anchor the threshold adds to": the lab jumps to the anchor that gets the most extra positives from the crop-wide threshold (the first, when several tie), and draws those extra positives hatched. Slide k, the negatives' per cent and the caps, and watch the histogram's two lines and the counts move. Real

5.4 The InfoNCE term, walked

Assumes: 5.2, softmax and temperature (Part 1), gradients (Part 1). Gives: the loss of one anchor, number by number, and the force it puts on the student's token.

For an anchor i and one of its positives p, the term asks the student's token of i to be closer to the teacher's token of p than to the teacher's tokens of all of i's negatives at once: it is near zero when the positive wins by a wide margin and grows when any negative comes close. Fact

In the lab below, the followed anchor's term is rebuilt from the real student and teacher tokens of run 2, step by step for one positive, and checked against the value that PyTorch computed for the bundle. Slide τ and watch the shares and the term; move the positive's cosine with the toy slider; click another anchor of the crop. Real

The gradient of the term is a sum of pulls and pushes on the student's token. The lab below draws them on a flat map of the crop's tokens and lets you take toy steps against the gradient. Toy

5.5 Why these choices

Assumes: 5.2 to 5.4. Gives: for each choice of the rule, the reason, the evidence, what it does to training and what changing it would do.

Most of the evidence below comes from the two premise tests: label-free measurements on existing weights, run on a CPU before the run, never a training run. They read the released model on the first 200 images of the image-only COCO v4 manifest. Fact

Several premise numbers use the frozen decoder's regions as a stand-in for objects: a region is a label-free guess, and a positive that shares its anchor's region probably lies on the same object. Fact

Before the run the second premise test also measured the combined rule at k = 8 on the released model: on the 224-pixel views 11.1 positives per anchor (median 8.0), 28.2 % of them from the threshold, the cap at 24 cutting 7.2 % of the anchors; on one draw of the trainer's crops 10.6 positives per anchor, 24.3 % from the threshold. Its verdict set τ = 0.2 and left k to the lead, who decided it before the protocol was committed. Fact

5.6 The projection head of run 3

Assumes: 5.4, a linear layer and GELU (Part 1). Gives: what run 3 changed, and why a head changes the gradient that reaches the backbone.

Run 3 starts where run 1 started and changes one knob: the space of the contrastive term, which moves into a projection head's outputs. Fact

The head is a Linear layer from 384 to 1,024 values with a bias, the exact GELU, and a Linear layer from 1,024 to 256 with a bias, applied to each token alone with no normalisation layer; its outputs are scaled to length 1 before the cosine. Fact

The positives, the negatives, both thresholds, k and the caps are still chosen from the teacher's raw final-block tokens, so the premise calibration and the rule's telemetry keep their meaning; only the term's similarities move into the head's space. Fact

The head moves the term's similarities into a token-wise MLP's space, as SimCLR applies its loss and DenseCL, Leopart and CrIBo their local terms, so that the push shapes the backbone through the head. Fact

What the head costs: two more modules to train and to carry, which add 10.5 MB to run 3's final envelope, the file in which a run saves its state, and a term whose negatives' share and margin are measured in the head's space, so they are not like for like with run 1's. Reading

5.7 The three runs, and the training in numbers

Assumes: 5.2, the recipe of Part 3. Gives: what each run tests, what it is compared with, and every number of its schedule.

Each further run changes one value against run 1, so that a difference has one cause. Fact

Each run completed its 9,240 steps with none skipped; run 1 trained for 35,521.3 seconds, run 2 for 35,435 and run 3 for 36,088, on one card. Reading

A run draws everything random from one number, its seed: the order in which the images come in each epoch, the crops of every image, the random numbers of every step and, from the DINO start, the first weights of the fresh heads. Two runs with the same seed see the same images in the same order with the same crops; a run with another seed sees other draws. Fact

Besides the starts themselves, two kinds of comparators already existed on disk. The two LC0 twins, named after LC0, the training recipe that they and all three runs share (its numbers are listed below), trained SelfPatch's own patch term from run 1's start, one with SelfPatch's row rule and one with the pooled image rule, with the same seed, initial state, image order, crops and per-step random seeds as run 1, so the contrasts with them are paired by construction. Fact

The pilot is the comparator of run 2 from the same DINO start: DINO trained with SelfPatch's window term under LC0 for 20 epochs, which had spent 0.56 points of the primary endpoint. Fact

The training, in numbers, for all three runs: Fact

• all 118,287 COCO train2017 images, images only, in batches of 256: 462 optimiser steps per epoch, the last 15 images of each epoch's order dropped;
• 20 epochs, so 9,240 steps; ten crops per image (2 × 224 + 8 × 96), so 174,080 anchors per step;
• the heads' learning rate peaks at 1e-4 and ends at 1e-6, the backbone's peaks at 1e-5 and ends at 1e-7, after a linear warm-up of 10 epochs and then a cosine;
• the backbone is frozen in epoch 0 (steps 1 to 462), so step 463 is the first in which the term can move it; the last layer of each head is held fixed in epoch 0 too;
• weight decay from 0.04 to 0.4 on a cosine; teacher momentum from 0.9995 to 1; the global term's teacher temperature 0.04 and student temperature 0.1; centre momentum 0.9; each gradient tensor clipped to 3.0; bf16 mixed precision, in which the backbones and the global heads compute with 16-bit numbers while the rule and the term are computed with 32-bit ones;
• pair 1 of the campaign's paired initial states, seed 906; an envelope every 5 epochs, and the final one, ckpt_epoch020.pt, is the object of the run: no checkpoint is chosen by any score.

The protocol laid the batch out as two cards with four micro-batches of 32 images each; the runs were executed on one card, card 6, with eight micro-batches of 32 images per step, a declared deviation that keeps the global batch at 256. Fact

5.8 How the model trains itself: the real telemetry

Assumes: 5.4 and 5.7. Gives: how to read every curve of a progress line, on the real numbers of the three runs.

The trainer writes one progress line per optimiser step with the step, the epoch, the losses, the learning rates, the weight decay, the teacher momentum and diagnostics; the contrastive path adds 14 fields, the term and 13 statistics of the rule. Fact

Read the curves of run 2 from left to right. At step 1 the total loss was 11.2743, the global term 10.5406 and the contrastive term 0.7336, and the negatives held 0.4976 of the denominators. By step 4,620 the term was 0.0372 and the share 0.0462; at step 9,240 the term was 0.0269, the share 0.0326 and the margin between a positive and the closest negative 1.2792. Real

Run 2's negatives' share fell fast at first, each epoch keeping about 75 % of the last, then about 87 %, and it sat near 0.03 for the last five epochs, above the stop line of 0.02. Reading

What the shapes mean. The term and the share fall together, because once the negatives are far their exponentials are tiny and the pulls and pushes weaken. The margin rises as positives and negatives move apart. τ_lo falls as the least similar pairs grow less similar and τ_hi rises as the most similar pairs grow more similar: the crop's thresholds move as the token space spreads. In run 1, τ_hi went from 0.63 to 0.73 and τ_lo from 0.11 to −0.06. Reading

5.9 What the readings showed

Assumes: 5.7 and its seed; the endpoints are explained in Part 6 and, one line each, in Lab 5G below. Gives: how to read a contrast and its interval, the decoder and probe numbers of every checkpoint, and what they do and do not say.

Each run's final envelope was read in one evaluation beside its comparators, by the frozen decoder on the COCO v4 validation panel: 1,000 images, 988 of them defined because 12 hold no retained truth object. A reading is a checkpoint's score on the panel, for most endpoints its mean over the defined images. A paired contrast is the mean, over the same images, of one checkpoint's value minus another's, written as that difference with its 95 % image-bootstrap interval in brackets. Fact

The interval asks how much the contrast could change if the panel had held other images of the same kind. The evaluator draws the panel's 1,000 images again at random with replacement, so that some come twice and some not at all, recomputes the contrast over the defined images drawn, and repeats this 2,000 times (500 times for AP100, the decoder's average precision); the 2.5 and 97.5 percentiles of the results, which hold the middle 95 per cent between them, are the interval's ends. Fact

A toy panel of four images, on which checkpoint b scores 1, −1, 3 and 1 points above checkpoint a, has the contrast (1 − 1 + 3 + 1) / 4 = 1 point. A resample that draws images 1, 1, 3 and 4 gives (1 + 1 + 3 + 1) / 4 = 1.5; one that draws 2, 2, 4 and 4 gives (−1 − 1 + 1 + 1) / 4 = 0; one that draws 3, 3, 3 and 1 gives (3 + 3 + 3 + 1) / 4 = 2.5. Over many resamples the results spread around 1, and their middle 95 per cent is the interval. An interval that leaves out 0 says that other images of the same kind would rarely turn the difference round; one that contains 0 says that they could. Toy

The interval holds one source of chance only, the images. A second source is the seed of 5.7: two runs that differ only in their seed end with different weights, and that seed spread is not in the interval of a run trained once. A verdict needs it: in the thesis's pre-registered selector study of 5.10, a contrast was declared different when the 95 % interval over the seeds left out zero and the mean difference exceeded a margin fixed in advance, equivalent when the 90 % interval over the seeds lay inside that margin, and inconclusive otherwise. Each contrast of a run here is a single-seed observation, so no verdict of different or equivalent is declared and nothing is adopted. Fact

Run 1, from the released start: on the primary endpoint it is indistinguishable from its comparators, +0.04 [−0.58, +0.63] points against the start, every interval about plus or minus 0.6 points around zero. Its term changed the decoder's output in one direction at every look: fewer background pixels claimed, a higher foreground union IoU, fewer regions, a lower foreground ARI and slightly lower recall. Reading

Run 2, from the DINO start, keeps the start's geometry: +0.18 [−0.59, +0.92] against the converted DINO, while the window-term pilot from the same start had spent 0.56 points. Against every comparator it claims much less background (13 to 15 points), raises the foreground union IoU by 4 to 5 points and the foreground ARI by 2 to 3, doubles AP100 and returns fewer regions, at the price of a little recall against its start. Under the rule the DINO start ends above the released start by +1.80 [+1.02, +2.61] points of Hungarian IoU. Reading

Run 3, through the head: +0.34 [−0.23, +0.93] against run 1 on the primary endpoint, and run 1's secondary signature is gone. Across the three runs, the rule on the raw tokens changes what the decoder groups, most from the DINO start, while the head version leaves the decoder where its start was. Reading

The linear probe tells another story. Run 1's tokens are more linearly separable than its start's by +1.35 mIoU [0.87, 1.79] while the decoder did not move; run 3 keeps that gain, 1.51 above its start; run 2 reads 4.0 mIoU below run 1 while the decoder ranks the two the other way round. The probe and the decoder reward different properties of the same tokens. Reading

The second seed pair has four runs from pair 2's initial states, seed 907: A repeats run 2 and B the window-term pilot from the DINO start; C repeats run 3 and D run 1 from the released start, all with the run-1 recipe and the rule's values. The amendment plans two evaluations after them, one for the DINO side and one for the SelfPatch side, with decision none, and the probes 9 to 12. With two seeds the thesis can say that a reading repeats in both seeds or in one of two, and no more than that. Fact

Runs A and C were launched on 12 October 2026 and completed their 9,240 steps on 13 October with none skipped; runs B and D then started on the same two cards. Reading

5.10 Before the rule: the selector study

Assumes: Part 4. Gives: the question the thesis asked first, what it found, and why it moved on to the rule.

The first question kept SelfPatch's loss and changed only how a patch picks its positives: one shared threshold instead of each patch's own top 4. SelfPatch trains each patch to agree with its 4 most similar neighbours; the thesis replaced that fixed 4 with one similarity threshold shared by a whole crop or a whole training step, hoping that the patches of one object would form islands. Fact

Three rules shared one form, the window neighbours above a threshold: the row rule (SelfPatch's own, the 4th largest of the anchor's 8 similarities), the image rule (one threshold per crop, the median of its similarities) and the batch rule (one threshold per optimiser step). Because they cut the same ranking, the only information a pooled rule adds is a comparison across anchors. Fact

The released-start campaign ran three pairs of row and image runs, seeds 906, 907 and 908. All four pre-registered verdicts of the sealed test reading on image minus row were "equivalent", the 90 % seed-level interval inside a margin of 1 point either side of zero, and the image arm met none of the quality bars. Fact

So far pooling has measured as weaker smoothing: the pooled arms sit where a row rule with k between 2 and 4 would sit, and no term of the loss lowers the agreement of any pair of patches. The report's analyses agreed that a separating term is needed for islands, though it is not enough on its own; this part adds one, a push, beside positives from the whole crop. Fact

5.11 Related ideas

Assumes: 5.1 and 5.6. Gives: where the rule's ideas come from, and what the reading of the literature found.

GLOM proposes islands of identical vectors as the parts of an image, and says that once two representations are sufficiently different there is no further pressure to push them even further apart; the course borrows the picture of islands, never a method. Fact

Run 3's projection head follows the way SimCLR applies its loss and the way DenseCL, Leopart and CrIBo apply their local terms, through a token-wise MLP. Fact

Our rule can be read as a loss that encourages islands of agreeing patch tokens inside one crop and pushes apart the tokens that disagree most. The link to GLOM is that picture and nothing more: the rule has no columns of vectors, no levels and none of GLOM's iterative settling. Reading

Two papers read on 12 October 2026

Two papers were read on 12 October 2026. What their readers found follows, as the lead's summary of their reports gives it. Reading

The Composer (ICLR 2024 submission 297, withdrawn; OpenReview JAfGlmRBTU) is a spiking network of two levels whose neurons synchronise in time: parts and wholes are read out as groups of neurons that fire together. It is trained with a denoising autoencoder for each level, on isolated single parts and wholes, with sixteen hand-set time constants, on four binary synthetic datasets (Ts, Squares, SHOPs and Double MNIST). Its coordination metrics need the true labels of the parts and the wholes, and its binary input gates every firing rate, so figure and ground never have to be separated. Reading

The readers' verdict: the Composer does not solve label-free grouping of photographs, and nothing in it carries over to our trainer. One idea survived, for evaluation only: a silhouette diagnostic of the instances in the frozen decoder's keys, pre-registered and used only to explain, which the owner has not taken up. Reading

Zeng, Song, Zhang, Hu and Yu, Agreement aware and dissimilarity oriented GLOM (ICCV 2025, pages 24351 to 24359) is a column network in GLOM's style that classifies images: SmallNORB, MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100. Its contrastive-attention operator is a step of the forward pass and no loss. In every update it splits all the pairs of columns at a level into an attract set and a repel set by a fixed cosine threshold, 0 at the bottom level and −0.2 at the top, and it moves each column toward the softmax-weighted mean of the columns that attract it and away from the columns that repel it. A pooling weighted by dissimilarity cuts its parameters from 72 M to 2 M. Its training is supervised, a supervised contrastive stage and then cross-entropy, and with those labels the paper reports a CIFAR-10 error of 10.90 against Agglomerator's 11.15. Reading

The readers found two defects in the paper's printed mathematics. Its dissimilarity metric reduces to at most 1/d for unit vectors of d numbers, so it cannot reach the published values of 0.37 to 0.54; and its negative term, which uses raw cosines as weights, moves the embeddings toward the negatives. Reading

What this network shares with our rule is the split, gated by a threshold, of all the pairs inside one image into pairs that attract and pairs that repel. Everything else differs: its operator sits inside the forward pass, it trains with labels, it works at the scale of CIFAR and it segments nothing. The readers kept three ideas for later: positives gated by a threshold, which our rule already has through τ_hi; loosening the positives as training goes on, judged risky; and one step of smoothing the agreement before the decoder, at low priority. Reading

What the literature sweep of 12 October 2026 found

The sweep looked for work after SelfPatch that resembles the rule. It read the citation lists of SelfPatch in Semantic Scholar, OpenAlex and OpenCitations, queried the arXiv API, searched OpenReview and read the title listings of CVF, ECCV, NeurIPS, ICLR, PMLR and TMLR, in English only and without Google Scholar. Reading

Within those sources it found no earlier work that trains a ViT backbone with a patch-level InfoNCE whose positives are the anchor's most similar tokens anywhere in the crop, a rank floor plus a crop-wide quantile threshold, and whose negatives are the least similar fifth of the crop, next to DINO's global term, and that reads the result with a frozen spectral decoder on class-agnostic COCO instance masks. Each piece exists somewhere; the combination was not found. This is a statement about the sources searched, and it does not prove that no such work exists. Reading

Two more works are the precedents for the way we evaluate: DiffNCuts (ECCV 2024) and SimDINO (ICML 2025) each read a short label-free fine-tune, or a changed self-distillation rule, with a frozen spectral decoder on COCO. Reading

The sweep also found evidence against parts of the design, which the thesis should report. Global-Local Self-Distillation (WACV 2023) found local positives mined by similarity harmful when the data are few. ASL (CVPR 2024) argues that a global contrastive term conflicts with patch contrast. VICRegL found a count of positives set by a threshold no better than a fixed count. AdaSim shows that nearest-neighbour positives without negatives can collapse, which is the argument for keeping negatives. Reading

LiFT (ECCV 2024) measured released SelfPatch below DINO on label-free TokenCut, with a COCO20K CorLoc of 52.18 against 53.98. That is the same direction as our decoder's Hungarian matched IoU on the validation panel in 5.9, 15.58 for released SelfPatch against 17.23 for the converted DINO. Reading

No published number is on our scale. The nearest are the Hungarian mIoU of the slot-attention line (7 slots, 256 px), mBO, FG-ARI, class-agnostic AP (bounded by recall for a decoder that returns few regions), CorLoc and semantic mIoU; our background false-positive rate has no published counterpart. Reading

This section quotes the lead's summary .superpowers/sdd/explainer_v2/related_work_summary.md, written on 13 October 2026 from the two paper reports and the consolidated sweep report of the day before; those three reports sit in the lead's working folder, outside the repository. Part 8 lists the summary with the course's other sources, in section 8.3. Fact