kparvataneni/memo-diff-jacobian-lens

MemoDiff Jacobian-lens matrices

Fitted per-layer transport matrices J_ℓ = E[dh_final / dh_ℓ] (4096×4096 fp32) used by the field Jacobian lens experiments in kveni12/model-memo-diff.

Layout

path	fit corpus	checkpoints
twin_2k_fineweb1k/step_*	FineWeb n=1000	2k PHI–benign SFT steps 120, 600
twin_2k_fineweb100/step_*	FineWeb n=100	same
uniform_public_n80/step_*	(run default)	uniform-public 3k step 1000

Each step_* directory contains:

• jacobians.pt — dict with metadata + jacobians: {layer:int -> Tensor[d,d]}
• field_jacobian_lens.json — per-field token-acc scores (when present)

Load example:

import torch
blob = torch.load("twin_2k_fineweb1k/step_600/jacobians.pt", map_location="cpu")
J = blob["jacobians"]  # layer -> (4096, 4096)