s/b

02 / RESEARCH NOTEBOOK

SEPTEMBER 2026

What I’m
working on.

I’m looking at how predictive models keep learning when the world changes, and what happens when their old representations stop working.

Reading and working through ideas.
Experiments are next.
QUESTION / 01CONTINUAL PREDICTION

Can a model recover
what it learned to ignore?

Say a model learns that color has nothing to do with motion. Later, color starts predicting friction. Can the model use that information again?

I’m looking at this in models that learn representations through prediction. First, I need to tell whether a feature disappeared from the representation or the predictor just stopped using it. Those are different problems.

I want to test whether a model can use that information again while keeping its earlier predictions useful, with the same memory and compute budget.

QUESTION / 02REPRESENTATION DRIFT

The representation
changed. Now what?

An encoder keeps learning, so its representation changes. What happens to a predictor trained on the earlier version?

I’m separating three possible causes: the coordinates changed, useful information was lost, or the underlying dynamics changed. All three can increase prediction error, but they need different fixes.

ENCODER → LATENT SPACE → PREDICTOR

Can useful predictions keep working as the encoder changes?

I’d test drift that comes from learning, alongside controlled changes like rotating the latent space.

QUESTION / 03MEMORY & PLASTICITY

What should
the model remember?

A model keeps receiving new data, but its memory is limited. Some rare experiences may matter long after it sees them.

I’m interested in how neural memory keeps useful information and updates knowledge that no longer holds. I also want to understand how to measure representation quality when both the encoder and the data keep changing.

I’d count stored observations, reference models and extra forward passes when comparing the cost of learning.

These are the questions I’m working through right now. I’ll put experiments and results here as the work develops.