Learning Without Weights: How CORE Distills Reasoning Insights From Contrast

Most of the recent progress in getting language models to reason better has come from throwing compute at the problem. Reinforcement learning from verifiable rewards (RLVR), exemplified by methods like GRPO, can produce real gains, but at a steep price: hundreds of thousands of rollouts and large training sets. Even the lighter-weight non-parametric alternatives, which leave model weights frozen and instead optimize prompts, still consume hundreds of training and validation samples. Humans, by contrast, often improve markedly after a handful of practice problems. The gap between these two learning curves is the motivating puzzle behind a paper I want to discuss in detail: CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning, by Nasvytis, Han, and colleagues at Stanford.

The central claim is straightforward and worth stating plainly. By comparing successful and unsuccessful reasoning traces and distilling the difference into short natural-language insights, a frozen model can improve faster and with fewer rollouts than both parametric (GRPO) and non-parametric (GEPA, episodic RAG, MemRL) baselines. With as few as five training samples, CORE achieves comparable or greater gains than each of those methods.

How CORE Works

CORE maintains two external memory stores rather than updating weights. The first, insight memory, holds generated insights: terse natural-language descriptions of strategies and constraints. The second, rollout memory, holds past attempts at problems along with whether they succeeded or failed.

The loop runs as follows. When the model encounters a problem, it first retrieves relevant insights and places them in context, then generates a reasoning trace and an answer. If the answer is correct, nothing dramatic happens. If it is wrong, CORE triggers its defining operation: it takes the failed trace and pairs it against a similar problem the model previously solved correctly, then prompts the model to contrastively reflect. The model is asked not simply to explain its mistake in isolation but to articulate what distinguishes the success from the failure. Insights that subsequently lead the model to solve the problem correctly are admitted to the insight store.

Two design choices deserve emphasis because they map cleanly onto the cognitive science the authors cite.

• Contrast over isolated reflection. Prior systems such as Reflexion let a model verbalize what went wrong on a single trajectory. CORE's bet, supported by work in human learning showing that people extract more general and reusable principles when comparing experiences rather than reflecting on them alone, is that contrast yields more transferable abstractions. The ablations in the paper confirm this contributes meaningfully to performance.
• Utility-aware retrieval. Insights are not retrieved on semantic similarity alone. Each insight carries a utility estimate updated with every associated success and failure. This is a small but consequential addition. It means the memory store is not just a growing pile of advice but a set of hypotheses that earn or lose credibility based on their track record. An insight that sounds plausible but never actually helps gets demoted.

The architecture deliberately echoes complementary learning systems theory, the idea that the brain has one system for encoding specific episodes and another for distilling general knowledge. Rollout memory is the episodic store; insight memory is the consolidated, abstract store.

What the Results Actually Show

The evaluation spans four reasoning tasks covering logic, planning, and problem solving. Two comparisons matter most.

First, under matched conditions CORE improves more rapidly than the baselines while spending fewer rollouts. This is the efficiency argument: the same or better accuracy reached after far less interaction with the verifier. Given that rollouts are the dominant cost in RLVR pipelines, a method that front-loads learning into a small number of well-chosen contrasts is genuinely valuable.

Second, and more striking, under fixed rollout budgets with as few as five training samples CORE matches or exceeds GRPO, GEPA, episodic RAG, and MemRL. The low-sample regime is where parametric methods typically fall apart; you cannot fit a meaningful gradient signal from five examples. That CORE remains competitive there suggests the mechanism is extracting something structurally useful rather than memorizing.

There is also a context-efficiency result that I find underappreciated. CORE stores compact insights rather than full reasoning traces or a single bloated global prompt, so it adds fewer prompt tokens at evaluation time than the non-parametric baselines. Episodic RAG approaches, by contrast, must inject entire prior trajectories into context, which scales poorly. Compression into abstract insights is doing double duty here: it improves both generalization and inference cost.

Finally, the interpretability angle is not incidental. The learned artifacts are human-readable strings paired with empirical utility scores. You can inspect what the model believes it has learned, which is something you cannot meaningfully do with a weight update. For anyone worried about opaque behaviors emerging from RL fine-tuning, this is a tangible safety and debuggability advantage.

My Take

What CORE gets right, in my reading, is the recognition that the bottleneck in fast learning is not the reward signal itself but the representation of what is learned. Scalar rewards are information-poor. A natural-language insight that says, in effect, "when the planning horizon exceeds three steps, enumerate constraints before committing to an order" carries far more transferable structure than a gradient nudge. The paper essentially argues that the right intermediate representation for cheap learning is abstract, verbal, and verifiable, and the evidence supports that.

The honest limitation is also clear, and it is the one I flagged in my earlier note: it is not yet obvious whether CORE is learning in a deep sense or performing very clever caching. If insights are mostly local heuristics tied to a narrow task distribution, then the gains, however real, are closer to retrieval than to abstraction. The five-sample result is encouraging because it implies generalization beyond the exact training items, but generalization within a task is not the same as transfer across tasks.

This is where I think the most interesting work lies next. The current memory store treats insights as an accumulating, lightly pruned set. I would push it toward an evolving population. Let insights compete on their utility estimates, prune those that stop predicting success, and actively abstract overlapping insights into higher-order principles. The question that would settle the learning-versus-caching debate is whether a small handful of meta-insights can generalize across tasks. If a principle distilled on logic puzzles measurably helps on planning problems, with no weight updates, that is transfer in the strongest sense available to a frozen model.

A second open question concerns the contrast-selection mechanism. CORE pairs a failure against a similar successful problem. But the most instructive contrast is not always the most similar one. Sometimes the sharpest lesson comes from a near miss that differs in exactly one decisive feature. A more deliberate, possibly counterfactual, selection of contrast pairs might yield richer insights, at some additional rollout cost. There is a tradeoff curve here worth mapping.

Conclusion

CORE makes a clean and well-supported case that distilling contrasts between successful and unsuccessful reasoning traces into compact, interpretable, utility-weighted insights is a more efficient and transparent route to self-improvement than weight updates, prompt optimization, or raw trace reuse. It is sample-efficient, rollout-efficient, context-efficient, and inspectable, which is a rare combination.

The framing I would keep in mind going forward is that CORE turns learning from verifiable rewards into a problem of knowledge curation rather than parameter estimation. That reframing opens a research agenda around how such knowledge should be represented, pruned, merged, and tested for transfer. The decisive experiment is still ahead: whether a small set of abstracted meta-insights can move performance across genuinely different tasks. If it can, the line between frozen-model retrieval and real learning will need to be redrawn.