Search Evolution Explorer

Level 8 · Learned ranking

src/domain/levels/level-3-learned.ts · 90 lines
Back to search

Let data choose the weights

Record which listing searchers clicked. Train a model that learns one weightThe number a feature is multiplied by when the score is built. Chapter 7 picks these numbers by hand; chapter 8 learns them from clicks. per featureOne measured property of a listing for a query, for example its BM25 score or its age. Features are the inputs of a learned model. from those clicks, so that the clicked listing scores above the skipped one. At query time every listing is scored on its own with the learned weights. The score has no unit; only the order matters.

Where to look: score computes the per-item featuresOne measured property of a listing for a query, for example its BM25 score or its age. Features are the inputs of a learned model. and multiplies each by its learned weightThe number a feature is multiplied by when the score is built. Chapter 7 picks these numbers by hand; chapter 8 learns them from clicks.. Nothing compares two listings at query time; the pairs live in the trainer.

Guided example

marblevs chapter 7

The learned weights lift an 'oriental marble look tile' into the top 10 for 'marble'; the hand-tuned boosts left it outside.

Run it hereOpen in compare

Prerequisites

Reuses the ranking fromChapter 7
Foundation · Inverted indexbuilt at start
Declared artifactsready
src/domain/levels/level-3-learned.ts90 linessame module the pipeline ran

Same candidatesAll listings that matched the query at all, before they are ordered. Scoring can only reorder this set. It cannot add to it. as chapter 7. A trained linear modelA score computed as weight 1 × feature 1 + weight 2 × feature 2 + ... The weights are learned from data instead of chosen by hand. replaces the hand-tuned sum: the same kinds of numbers become featuresOne measured property of a listing for a query, for example its BM25 score or its age. Features are the inputs of a learned model., and data chooses the weightsThe number a feature is multiplied by when the score is built. Chapter 7 picks these numbers by hand; chapter 8 learns them from clicks..

import { level3Content } from "../../content/levels/level-3.ts";
import type { Clock } from "../ports/clock.ts";
import { , } from "../ranking/features.ts";
import { } from "../ranking/linear-scorer.ts";
import { } from "../signals/build-signals.ts";
import { } from "../signals/frozen-clock.ts";
import type { EvolutionLevel, LevelDeps, LevelRanker } from "../types/level.ts";
import type { QueryUnderstanding } from "../types/query.ts";
import { as } from "./level-2-boosting.ts";
/** Duplication across levels is accepted so each evolution level stays independently readable. */
export function createRanker(deps: LevelDeps): LevelRanker {
const previous = (deps);
// Recency depends on the time. One query reads the clock once, so two listings posted at
// the same moment score exactly the same, and a millisecond tick while scoring the
// candidates cannot change their order.
const clockByQuery = new WeakMap<QueryUnderstanding, Clock>();
return {
match: previous.match,
score(understanding, candidate) {
const features = (understanding, candidate, {
index: deps.index,
levers: deps.levers,
clock: (understanding, deps.clock, clockByQuery),
});
// The model's bias (its intercept) is the same for every candidate, so it never changes the
// order. It is left out of the reported score so the number a learner sees is the part that
// actually moves listings, and it is not shown as a signal that is identical for every row.
const learned =
deps.model === undefined ? 0 : (deps.model, features) - deps.model.bias;
const weights = deps.model?.weights ?? [];
return {
item: candidate,
score: learned,
signals: (understanding, candidate, [
{ name: "learnedScore", value: learned },
...FEATURE_NAMES.map((name, index) => ({
name,
value: features[index] ?? 0,
weight: weights[index],
})),
]),
};
},
};
}
function frozenClock(
understanding: QueryUnderstanding,
clock: Clock,
cache: WeakMap<QueryUnderstanding, Clock>,
): Clock {
const known = cache.get(understanding);
if (known !== undefined) {
return known;
}
const frozen = (clock);
cache.set(understanding, frozen);
return frozen;
}
a numbered note, explained on the righta function you can open heredefined further down in this filewhere level 7 joins