CASE 02 — MEMORY & CONTEXT
A tutoring session outgrows the context window, and the naive fix — summarise the old messages — throws away the one thing worth keeping. Extract before you compress.
Fig. 2 — Compaction pipeline
A student can work a problem set for an hour. The history grows past what fits, and something has to go — but a tutoring transcript is not a chat log you can discard. Buried in it is the thing that makes the next session better: what this student actually finds hard.
Compaction runs in that order deliberately. First a pass over the old messages pulls out durable memories — what the student struggles with, what has clicked, how they prefer to be shown things — and persists them to Firestore against the student, not the session.
Only then are those messages summarised into a single compressed message, with the five most recent turns kept at full fidelity so the immediate thread of the conversation is untouched.
Extraction, summarisation, quality scoring and classification all run on Haiku. None of them are the teaching task, and paying the teaching model's rate to decide whether a memory is worth keeping is how an AI product quietly becomes unaffordable.
Memories also go through a quality filter and a consolidation pass, so the store doesn't accumulate fifty near-duplicate observations that a student finds fractions hard.
Retrieval is Firestore native vector search: the embedding is a Vector field on the memory document itself, and similarity search runs through find_nearest() against the same documents the rest of the application reads.
The alternative — a dedicated vector database alongside Firestore — means two systems that can disagree about what a student knows, plus a sync path that fails quietly. One store is simpler and has one fewer way to be wrong.
Anthropic retired claude-3-haiku-20240307. The direct API began returning a 404, and because extraction and summarisation were both wrapped to never break a student's session, compaction kept running and silently produced nothing — no memories extracted, no summary written, for every compaction, for as long as it took to notice.
That is the hazard of defensive error handling: the same wrapper that stops a background task from ruining a lesson also stops it from telling you it has stopped working.
Fixed to the current Haiku, with the incident written into a comment on the model constant so the next person to touch it knows why it is pinned and what to check.