PDE-OBS / README.md

Card: E6 weights and the path D retrains (October 2026)

ab1197f verified 4 days ago

10.2 kB

license: mit
tags:
  - partial-differential-equations
  - scientific-machine-learning
  - neural-operators
  - partial-observations
  - benchmark

PDE-OBS: trained baseline checkpoints

Final checkpoints of the PDE-OBS campaign: one model per (PDE family, method, training observation pattern). Each checkpoint is the final recorded checkpoint of its attempt, not one selected on validation or test error.

All 441 credited settings are published here. models_manifest.<cluster-label>.json lists the SHA-256 of every published file. The physical records these models were trained and scored on are in the companion dataset ru1ch3n/PDE-OBS.

Layout

models/<cluster-label>/<pde>/<method>/<train_view>/
    checkpoints/last.pt              final checkpoint: weights, scheduler state, history, resolved config
    checkpoints/training_config.json resolved trainer configuration
    identity.json  provenance.json  resolved.yaml  split_manifest.json  factor_coverage.json
    completion.json  health.json  history.json          (where the attempt wrote them)
    budget-protocol.json | dynamic-protocol.json        (cohort dependent)
models_manifest.<cluster-label>.json          per-file SHA-256 of the published files (repository root)
scrub-manifest.json                           which record files the last de-identification pass rewrote

The cluster labels (cluster-A, cluster-B, cluster-C) stand for the three machines the campaign ran on; they carry no institutional meaning. resolved.yaml beside each checkpoint records the architecture (method.name, method.kwargs), the training view and the training settings.

Scoring a checkpoint with the benchmark

from pdeobs import api

data = api.load_dataset("./pdeobs-data", verify=True)              # from the dataset ru1ch3n/PDE-OBS
package, targets = api.inference_input_from_dataset(data, api.make_observation("paper:R50"), task="recovery")

pred = api.load_legacy_checkpoint("models/cluster-A/poisson/fno/random_50pct/checkpoints/last.pt",
                                  model={"name": "fno", "preset": "paper"}, task="recovery")
score = api.evaluate(api.predict(pred, package), targets)
score["status"], score["scoring_version"], score["summary"]["rel_l2_joint_mean"]

The loader takes the structure from the caller (the campaign checkpoints use the paper preset of their method), cross-checks the checkpoint's own stored training configuration, and records the file's SHA-256 and epoch in the predictor's provenance. Without the benchmark, the file is a plain PyTorch payload: torch.load(path, map_location="cpu", weights_only=False) returns a dict with model_state, config, history, epoch and scheduler_state.

Important properties

Weights only. Optimizer, gradient-scaler and RNG states were removed (about two thirds of each file); dropped_keys in the manifest records this per checkpoint. The checkpoints support inference and rescoring, not exact continuation of training.
De-identified metadata, unmodified parameters. Every string naming a filesystem path, account, node, host, login node, scheduler job, partition, GPU hardware identifier, interpreter or environment directory, run directory or source-code revision was rewritten to a placeholder such as <ROOT_A>, <USER>, <NODE>, <JOB>, <PYTHON>, <ENV>, <RUN> or <COMMIT-1>, and raw scheduler records were reduced to their resource fields (<SCHEDULER_RECORD> ...); three passes, the files rewritten by the last one are listed in scrub-manifest.json. Model parameters were not modified. models_manifest.*.json records the SHA-256 of each released file; the correspondence to the campaign's checkpoint identities is kept with the benchmark code.
Training cohorts are not interchangeable. Each entry carries its cohort label: the original 500-epoch protocol, a recovered continuation of it, a declared 200-epoch budget, a budgeted max-200 patience protocol, or a salvaged interrupted attempt. Any table that pools cohorts must say so per row.
Scoring provenance. All 441 checkpoints were later rescored from retained prediction arrays with the benchmark's strict scorer (pdeobs-strict-v1, nine views x 200 held-out identities each); those results and their per-identity errors accompany the benchmark code (results/prediction_verification_20260924/). The campaign's own evaluator metrics remain in the archived-results index and are not the paper's current numbers.

October 2026 revision models (`october2026/`)

The trained models behind the October 2026 revision results of the benchmark code (results/revision_october/, frozen recipes in configs/paper/october/recipes.yaml): one entry for every trained run that has scored test views in those tables, 663 runs in the cohorts P4, P5, P6, P7, V4C, V4D, V4E, V4P6, V4P7, N3GJ2, N5MXR6, N6, N8, RPB, RPBn and E6 (the 12 E6 runs, nearest-fill copies of zero-fill FNO runs, were added on 2026-10-21 as upload part 3). Runs that failed during training (they are listed with their failure in the result tables) and the recipe-screen runs have no model here. The training and test data are the V2 views in the companion dataset, under october2026/.

october2026/models/<cohort>/<run_id>/
    weights.pt     {"format": "pdeobs-october2026-weights/1", "run_id": ..., "model_state": <state dict>}
    model.json     cohort, PDE, training view, method, protocol, seed, grid, epochs, final training loss,
                   learning rate, number of training records, recipe digest, original and released SHA-256
    history.json   per-epoch training history (numeric fields only)
october2026/models_manifest.part-<n>.json   the model.json records of each upload part

<run_id> is <cohort>__<pde>__<ell>__<method>__<protocol>[__g<grid>]__s<seed>, the run identifier used in the result tables.

Weights only. model_state is the state dict of the run's final checkpoint, unchanged; where a run used the training-set normaliser, it is part of the state dict (buffers v4_mu / v4_sigma). Optimizer, scheduler, RNG state and the resolved run configuration (which held machine paths) are not included; original_checkpoint_sha256 in model.json is the digest of the full checkpoint the campaign recorded at the end of training, checked against the file before the weights were extracted. torch.load(path, map_location="cpu", weights_only=True)["model_state"] loads it.
Rescoring. For the cohorts with a frozen recipe in configs/paper/october/recipes.yaml (P4, P5, V4C, V4D, N3GJ2, N5MXR6, N6, RPB, RPBn, E6), reproduce/october/replay_inference.py --run <run_id> --checkpoint weights.pt --data-root <download>/october2026/v2 rebuilds the network, re-predicts the run's test views and compares each view mean with the stored score (relative tolerance 1e-4; recorded checks on CUDA and CPU are in reproduce/october/README.md; an E6 run, Poisson rough G256 seed 1101, replayed on a CPU matches its stored scores to 2e-7). V4E is listed in recipes.yaml as four information cells that the replay script does not rebuild; P6, P7, V4P6, V4P7 and N8 have no entry (they reuse the P4 or V4C recipe of their slice with one change: training-set size, noise, input fill, mixture or grid). The FNOs were trained and scored with cuFFT; on a CPU the script reproduces cuFFT's large-batch inverse real FFT (pdeobs.revision.fftcompat).
Checked after upload. Every weights.pt on the Hub has the SHA-256 listed in its manifest part (checked without credentials).
RPB / RPBn models were trained on the Poisson part of the CNO authors' Representative PDE Benchmarks (Raonic et al.; Zenodo record 10406879, CC BY 4.0). That data is not redistributed here; to rescore these models, download it from Zenodo and convert it with python -m pdeobs.revision.external.rpb.

Path D retrains (`october2026/retrains/`)

Ten retrains of Poisson medium U256 runs from the public package with reproduce/october/train_run.py of the benchmark code (reproduction path D, 2026-10-18 to 2026-10-21): P4 FNO seed 1102, P4 Voronoi-CNN seeds 1101-1103, V4C FNO seeds 1101-1103 and P4n FNO seeds 1101-1103. P4n is the P4 recipe with --set v4_norm=train_scalar: the training-set normaliser is its only change. The P4 FNO retrains of seeds 1101 and 1103 collapsed to a constant output, as the original runs did, and have no model.

october2026/retrains/<name>/
    weights.pt     same format as the October models
    model.json     recipe run, --set overrides, final training loss, normaliser, the seven v2eval7 view means of the
                   retrain, original and released SHA-256
    history.json   per-epoch training history (numeric fields only)
october2026/retrains/manifest.json   the model.json records, and the two retrains without a model

These are retrains, not the campaign's models. Retraining is not bit-identical, so their scores differ from the stored scores of the original runs; results and discussion are in reproduce/october/README.md of the benchmark code. Rescore a retrain with its recipe run and the same --set (on a CPU, the P4n seed 1101 replay reproduces the view means in its model.json to 9e-7):

python reproduce/october/replay_inference.py --run P4__poisson__ell009__fno__U256__s1101 --set v4_norm=train_scalar \
    --checkpoint october2026/retrains/P4n__poisson__ell009__fno__U256__s1101/weights.pt --data-root <download>/october2026/v2

Citation

Ruichen Xu, Siyao Wang, Fang Wan, Jiacheng Qiu, Wenhan Gao, Jiaxing Zhang, Linsey Pang, Ravid Shwartz-Ziv, Yann LeCun and Yuefan Deng. PDE-OBS: Controlled Evaluation Across Observation Patterns. arXiv preprint arXiv:2609.36521, 2026.

@article{xu2026pdeobs,
  title   = {{PDE-OBS}: Controlled Evaluation Across Observation Patterns},
  author  = {Xu, Ruichen and Wang, Siyao and Wan, Fang and Qiu, Jiacheng and Gao, Wenhan and Zhang, Jiaxing and
             Pang, Linsey and Shwartz-Ziv, Ravid and LeCun, Yann and Deng, Yuefan},
  journal = {arXiv preprint arXiv:2609.36521},
  year    = {2026}
}